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ABSTRACT. Motivated by S-duality modularity conjectures in string the- 
ory, we study the Donaldson-Thomas type invariants of pure 2-dimensional 
sheaves inside a nonsingular threefold X in three different situations: 

1. X is a K3 fibration over a curve. We study the Donaldson-Thomas 
invariants defined in ] ThoOO [ of the 2-dimensional Gieseker stable 
sheaves in X supported on the fibers. Analogous to the Gromov- 
Witten theory formula established in |MP07|, we express these in- 
variants in terms of the Euler characteristic of the Hilbert scheme of 
points on the K3 surface and the Noether-Lefschetz numbers of the 
fibration, and prove that the invariants have modular properties. 

2. X is the total space of the canonical bundle of P 2 . We study the 
generalized Donaldson-Thomas invariants defined in | JS11 [ of the 
moduli spaces of the 2-dimensional Gieseker semistable sheaves on 
X with first Chern class equal to k times the class of the zero sec- 
tion of X. When k = 1, 2, or 3, and semistability implies stability, 
we express the invariants in terms of known modular forms. We 
prove a combinatorial formula for the invariants when k = 2 in 
the presence of the strictly semistable sheaves, and verify the BPS 
integrality conjecture of |JS11| in some cases. 

3. (Joint with Richard Thomas) X is a Calabi-Yau threefold and L is a 
sufficiently positive line bundle. We define new invariants count- 
ing a restricted class of 2-dimensional torsion sheaves, enumerating 
pairs Zc HinX. Here H is a member of the linear system \L\ and 
Z is a 1-dimensional subscheme of it. The associated sheaf is the 
ideal sheaf of Z C H, pushed forward to X and considered as a cer- 
tain Joyce-Song pair in the derived category of X. We express these 
invariants in terms of the MNOP invariants of X |MNOP06|. 
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0. Introduction 

0.1. Overview. We study the invariants virtually counting the configura- 
tions of a number of points and a vector bundle supported on the mem- 
bers of a system of divisors inside a nonsingular threefold. One of our 
motivations is that these invariants have been studied by the physicists 
BGSY071 IGY071 IOSV01| as a set of supersymmetric BPS invariants associ- 
ated to D4-D2-D0 systems. By string theoretic considerations the generat- 
ing series of these invariants are expected to be modular. 

In this paper we interpret these invariants in terms of the moduli spaces 
of pure coherent sheaves with 2-dimensional support inside a smooth three- 
fold X. Another motivation for considering pure 2-dimensional sheaves is 
to find a sheaf-theoretic analogue of the formulas proven in [MP07J that 
relate the Gromov-Witten invariants of a threefold to the Gromov-Witten 
invariants of a system of its divisors. 

Purity of a 2-dimensional coherent sheaf T means that all the nonzero 
subsheaves of T have 2-dimensional supports. To construct a moduli space 
for the pure sheaves (possibly with additional structures) one needs a no- 
tion of stability. For the objects in this article we consider two types of 
stability: Gieseker stability and the stability of pairs l|Pot93l|jSTT] . 

Definition 0.1. Let (X, O (1 ) ) be a smooth threefold with a polarization. For 
a pure sheaf J 7 the Hilbert polynomial is defined to be Pjr(m) = ^(X,J r (m)), 
and the reduced Hilbert polynomial of T is 



fj- = Pjr /leading coefficient of Pjr. 
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• T is called Gieseker semistable if for any subsheaf Q C T we have 
VQ i m ) < V^{ m ) f° r ^ 2> 0. J 7 is called Gieseker stable if the equal- 
ity never holds for any proper subsheaf Q. 

• For a fixed n >0a pair ( T , s), where s is a nonzero section of T(yi), 
is called stable if 

(1) J 7 is Gieseker semistable, 

(2) if s factors through a proper subsheaf Q{n) then pg(m) < pj{m) 
for m > 0. 

Suppose now that M. is a proper moduli space of Gieseker stable sheaves 
J 7 or the moduli space of stable pairs (Fj s) as above with fixed Hilbert 
polynomial Pj. Ai is usually singular and may have several components 
with different dimensions. To define (deformation invariant) invariants as 
integration over M. we need to have a virtual fundamental class of the moduli 
space constructed by means of a perfect obstruction theory on Ai . This can 
be obtained by studying the deformations and obstructions of the stable 
sheaves or the stable pairs HBF971 ILT981 IThoOOl IHT10II . 

One of the richest examples of a case that the moduli space of sheaves 
over the threefold X admits a perfect obstruction theory is the moduli space 
of ideal sheaves of 1-dimensional subschemes in X which can be identi- 
fied with a component of the Hilbert scheme of curves in X ([M NOP061 
ThoOOJ). When X is a Calabi-Yau threefold, then the obstruction theory 
on the Hilbert scheme is symmetric and the corresponding invariants are 
expressible as a weighted Euler characteristic of the moduli space [Beh09J. 
The invariants are denoted by Ia n depending on a curve class /3 in X and an 
integer n keeping track of the holomorphic Euler characteristic of the ideal 
sheaves. These invariants conjecturally determine all the Gromov-Witten 
invariants of X (see |MNOP06|). Neither set of invariants are expected to 
have modular properties for general X. 

Given this overview, we study the invariants arising from the pure 2- 
dimensional sheaves in three different situations depending on the geome- 
try of X and what stability condition we impose. We are able to prove the 
modularity predicted by physicists in some cases. In the rest of this section 
we explain each situation separately and state our main results. 



0.2. DT invariants of 2-dimensional sheaves. Let X be a smooth projec- 
tive threefold over C with a fixed polarization L. Let P(m) be a degree 
2 polynomial. We consider the moduli space Ai = Ai(X;P) of Gieseker 
semistable sheaves with Hilbert polynomial P. By the assumption on the 
degree of P, the support of any C-point of Ai is 2-dimensional. It is proven 



in |Tho00| that Ai admits a perfect obstruction theory if the following con- 



ditions hold true: 

• There are no strictly semistable sheaves with Hilbert polynomial P. 

• Ext 3 (J 7 , J 7 ) o = for any stable sheaves with Hilbert polynomial P 
(the index indicates the trace free part of Ext 3 (J-", J 7 )). For stable 
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sheaves on any threefold with effective anti-canonical bundle this 
condition is satisfied by Serre duality. 

Definition 0.2. [ThoOO] Suppose that the conditions above hold true for 
M.. Then HThoQOH constructs a natural perfect obstruction theory E* — > L' M 
such that 

h\E' v ) = £xi^(F,F) 

for i = 0,1. The corresponding virtual cycle [.A4] 1 "'' = [M.,E m ] mr has di- 
mension d = ext 1 (J r , J 7 ) — ext 2 (J-", J 7 ), d is called the virtual dimension of 
A4. The Donaldson-Thomas invariants are then defined by integrating a 
class ol e H 2d (M,Q) against [M] vir : 

DT(X;P;a) = f a. 

J\MY ir 

When the virtual dimension is zero, we define DT(X;P) = DT(X; P; 1) to 
be the degree of [.M] Bir . The invariant DT(X; P) is always an integer. 

Example 0.3. (2-dimensional sheaves on P 3 ) Let X = P 3 with polarization 
L = 0(1), and 

P(m) = m 2 /2 + (s + 3/2)m + d 

for some s,d 6 Z. Then Ai = .A4(P 3 ;P) is the moduli space of semistable 
sheaves supported on the hyperp lanes in P 3 . Denote the hyperplane class 
by H. Note that since Ci(T) = H is an irreducible class, then there are 
no strictly semistable sheaves with Hilbert polynomial P. Moreover, since 
K x = -4H, Serre duality and the stability of T imply that Ext 3 (J 7 , T) = 0, 
and hence M. is equipped with a perfect obstruction theory. In Section[l]we 
show that the virtual dimension of M. is always 3 independent of s and d. 
Furthermore, if a is the pull back of the class of a point from (P 3 ) v (under 
the natural morphism A4 — > (P 3 ) v taking a sheaf to its support) then 

(1) DT(X;P; a ) = / c top (S), 

JHilb"(P 2 ) F 

where Hilb"(P 2 ) is the Hilbert scheme ofn = s(s + 3)/2 — d + 1 points on 
P 2 and S is a locally free sheaf of rank In on Hilb"(P 2 ). We will give a 
combinatorial formula for this integral in Section [l] 

In case X is a K3 fibration over a smooth curve, we study the moduli 
space A4(X;P) of sheaves supported on the K3 fibers. Motivated by the 
ideas of |MP07] in Gromov-Witten theory, we express DT(X; P) in terms of 
the Euler characteristic of the Hilbert scheme of points on the K3 surface 
and the Noether-Lefschetz numbers of the fibration IIMP071 |Bor99l |Bor98l 
KM90|. Using the degeneration techniques of Wu and Li [LWllj and de- 
formation invariance of the DT invariants, we extend this formula to the 
case where finitely many of the K3 fibers have rational double point sin- 
gularities. In Section [2] we put the invariants DT(X;P) into a generating 
function Z(X,q) by letting the formal variable q keep track of the constant 
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term of P. We completely determine the generating function Z(X,q) and 
write it in terms of some known modular forms: 

Theorem 1. Let X be a K3 fibration over a smooth curve with at most finitely 
many of the fibers having nodal singularities. Let M.{X;P) be a proper moduli 
space ofGieseker stable sheaves supported on the fibers. Assuming that the linear 
term ofP is nonzero then 

where O(^) is a vector valued modular form keeping track of the Noether-Lefschetz 
numbers of a nonsingular model ofX, and rj(q) is the Dedekind Eta function. 

In the future we plan to use similar degeneration techniques to find the 
DT invariants of quintic Calabi-Yau threefolds (and other Calabi-Yau com- 
plete intersections) corresponding to 2-dimensional sheaves supported on 
hyperplane sections following ideas similar to those in [MP06a. PP12J. We 
hope to find a more general explanation of the modularity of the DT invari- 
ants of the quintic predicted by physicists IIGSY071 iGYOTl IOSV0 1 II . 

0.3. Generalized DT invariants of 2-dimensional sheaves. Let X be a Ca- 
labi-Yau threefold and let P(m) be a degree 2 polynomial. If the moduli 



space of Gieseker semistable sheaves M.{X;P) (as defined in Section 0.2 1 



contains strictly semistable sheaves then one cannot define the invariants 



of M (X; P) by the methods of Section 0.2 Joyce and Song [JS11 1 instead de 



fine Q-valued invariants for M(X; P) called the generalized DT invariants 
DT(X;P) by considering the contributions of strictly semistable sheaves 
using sophisticated stack functions. The generalized DT invariant is special- 



ized to DT(X;P) of Section 0.2 if there are no strictly semistable sheaves 
and moreover, DT(X;P) is also deformation invariant. 

We study in detail the case where X is the total space of the canonical 
bundle of P 2 and M.(X;P) is the moduli space of semistable sheaves with 
Hilbert polynomial P{m) = rm 2 /2 + . . . . Any semistable sheaf T with 
Hilbert polynomial P is (at least set theoretically) supported on the zero 
section of X, and C\ (J 7 ) is equal to r times the class of the zero section. We 
relate DT(X; P) to the topological invariants of the moduli space of torsion- 
free semistable sheaves on P 2 . Using the wall-crossing formula of Joyce- 
Song IJS11J and the toric methods of IPe r04llKoo0 8l we find a formula for 
DT(X;P) when r = 2 in the presence of strictly semistable sheaves. To ex- 
press the result, let M.{F 2 ;P) be the moduli spaces of Gieseker semistable 
rank 2 sheaves on P 2 with Hilbert polynomial P and let .A4 S (P 2 ; P) be the 
open subset of stable sheaves. Denote by Hilb" (P 2 ) the Hilbert scheme of 
n points on P 2 . Then we prove 

Theorem 2. Let P(m) = m 2 + 3m + b where b G 2Z then 

DT(X;P) = X mb- b/2 (F 2 ))/4- x(M 5 (F 2 ;P)) - C(b) 
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where C(b) is a combinatorial expression depending only on b and is defined in 
Section^(see Definition 3.7). 



0.4. Ideal sheaves on members of a linear system^] 

Let X be a projective Calabi-Yau threefold such that H 1 (X, Ox) = 0, and 
fix a complete linear system | H | on X which is sufficiently positive in the 
sense of Definition |R2] Let 

Z C H 

be a 1-dimensional subscheme of a member of this linear system. The ideal 
sheaf of Z is a torsion-free rank 1 sheaf Iz on H. Letting i : H <— » X be 
the natural inclusion, ulz is a pure 2-dimensional sheaf in X. Suppose 
that c\\2(UOz) = f> and x{®z) = n, and let H — > \H\ be the universal 
hyperplane. Then there exists a relative Hilbert scheme 

(2) Hi% n (ft/|H|) 

parameterizing these pairs Z C H in X. We would like to consider it as 
parameterizing the coherent sheaves i*Iz, but they may not be Gieseker 
stable (if H is reducible or not reduced) and there may be deformations of 
U lz which are not themselves pushforwards of ideal sheaves. 
Notice that i* lz is the cokernel of the map 

0(-H)^l z , 

where sh is the section of O(H) vanishing on H, and Iz is the ideal sheaf 
of Z when considered as a subscheme of X (rather than H). Therefore i*lz is 
quasi-isomorphic to the complex 

where we put Iz in degree 0. If we consider Sh to be a section of Iz (H) then 
{Iz,Sh) is an example of a Joyce-Song pair [JSllj, and it is automatically 



stable as a pair in the sense of Definition 0.1 So pair stability defines a 
different notion of stability on the sheaves i*Iz that the complexes I' are 
quasi-isomorphic to, with respect to which all of these torsion sheaves are 
stable. 

For H sufficiently ample, Joyce and Song show by deformation theory 
that the moduli space of their stable pairs is a locally complete moduli 
spac^Jof objects I' of D(X). In our situation this means that Hilb^ / „('H/|H| ) 
is a locally complete moduli space of sheaves i*lz- In other words, the 
sheaves U iz do not deform to sheaves of a different form, for H sufficiently 
ample. 



This section is joint work with Richard Thomas. 
2 In the sense of IHT101 Section 4.1]. 
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Theorem 3. Let X be a nonsingular projective Calabi-Yau threefold such that 
H 1 (X / Ox) = 0. Given a curve class f> € H2(X;Z) and n S Z, let H be suffi- 
ciently positive with respect to [5, n in the sense of Definition B.2 Then 

mbfa(H/\H\) 

is a locally complete moduli space of torsion sheaves on X. It admits a symmetric 
perfect obstruction theory and so a virtual cycle of virtual dimension 0. 

We can therefore define an invariant 

p ' 7[Hi%„(-H/|H|)r 

Since the obstruction theory is symmetric, by [Beh09| this can also be writ- 
ten as a weighted Euler characteristic of Hilb^ „('H/ |H|). The invariants 
N^ n are closely related to the MNOP invariants Ip /fI counting ideal sheaves 
Xz'ofX. 

Theorem 4. Under the conditions ofTheorem^we have 

^„ = (-ir l c-^ M/ 

where c = c(/3, n, H, X) is the topological number 

J x (^H 3 + H 2 td 2 (X)-H-^-n. 

Example 0.4. Most of the examples worked out explicitly in HGY07 , Sec- 
tions 2.1 to 2.6] fit into the setting of this section and hence, the correspond- 
ing invariants are captured by the invariants Nj^ n . As an illustration, we 
work out the following two simple cases for linear hyperplane sections H 
of the quintic Calabi-Yau threefold X. 

• Ip is the ideal sheaf of a point p G X, then /3 = 0, and n = 1, and 
hence 

c = td 2 (X) • H + H 3 /6 - 1 = 25/6 + 5/6-1=4. 

Also J ,i = -x(X) = 200, and hence by Njft = (-1) 3 x 4 x 200 = 
-800. ' 

• Xq is the ideal sheaf of a line C C X, so j6 • H = 1, n = 1, and hence 

c = 25/6 + 5/6-1 =3. 

Also la i = 2875, the number of rational degree 1 curves in a generic 
quintic, and hence N? x = (-1) 2 x 3 x 2875 = 8625. 
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1. Proof of Formula ^ 



In this section we continue with the set up and notation of Example 0.3 
We first prove ([TJ, and then use torus localization to evaluate the integra 
appearmg m 0. For any C-point T of M = M(F 3 ,P) we know that 
C\ (X) = H. Therefore, by Hirzebruch-Riemann-Roch formula 

= I ch v (X) • ch(J-) • td(P 3 ) = 

JF 3 

[ (H - ch 2 (X) + ch 3 (X)) • (H + ch 2 (X) + ch 3 (X)) • (1 + 2H+ . . . ) = 

/ -2H 3 = -2. 
Jw 3 

and hence the virtual dimension of Ad is 3, independent of s and d. We 
denote the tangent and the obstruction sheaves of M. by T and Ob respec- 
tively. 

We have a natural morphism n : A4 —> (P 3 ) v = P 3 which, at the level 
of C-points, sends a sheaf T to its support. For any x G P 3 the fiber of n 
over x is isomorphic to the Hilbert scheme of n points on P 2 , denoted by 
Hilb"(P 2 ), where 

n = s(s + 3)/2 -d + 1. 

A C-point T in this fiber is then the push forward of I(s) := X ® O^iis) 
via the inclusion of the hyperplane P 2 P 3 , where X is the ideal sheaf of 
some length n subscheme of P 2 . The fibers of T and Ob over the C-point 
J 7 fit into the exact sequence (see [ThoOO. Lemma 3.42]) 

-> TxHilb^P 2 ) -> 7> -> Hom(X,X(l)) -> -> Ob^ -> -> 0, 

where f is a locally free sheaf of rank 2n over Hilb"(P 2 ) whose fiber over 
X is identified with Ext p2 (X,X(l)). From this description one can see that 
M. is smooth of dimension 2n + 3 and moreover, Ob is locally free of rank 
In. So by [BF97, Proposition 5.6] we have 

[Mf r = c 2n (Ob) n [M\. 
Now if [pt] is the class of a point on P 3 then 

DT(P 3 ;P;7r*[pf]) = f 71* [pt] 

J{M] vir 

= f c 2n (Ob)\J7i*[pt] = f c 2n {£). 

J[M] y[Hilb"(P 2 )] 
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The last integral can be evaluated by localization method. C* 2 acts on P 2 
via 

(h,h) ■ (*i : x 2 : x 3 ) = (x 1 : t t x 2 : t z x 3 ). 

It is well-known that (see |ES87[|Nak99]| ) under the induced action on Hilb" (P 2 ) 
the fixed point set is in bijection with the set of triples of 2d partitions as 
follows: 

Fix„ = {n = (m, 7T 2 , 7T 3 )| ^#7T, = n) . 

Now given a 2d partition n, suppose b £ tx is a box in the Young dia- 
gram associated to n. Let a (b), 1(b) be respectively the number of boxes 
above b and on the right side of b in the Young diagram of n. Now given a 
fixed point Q corresponding to (n\, n 2 , n 3 ), the C* 2 -character of the tangent 
space Tq Hilb" (P 2 ) is given by 

Y t m+l t -«(b) +f -I(6) f a(b)+l + 

^ t a(b)-l(b)-l t -a(b) +f /(6)-«(fc)-l f fl(6)+l + 

be7z 2 

f «(fc)-/(6)-l f -«(fe) +f W-«(fc)-l f fl(6)+l 

By a simple modification of the proof of the statement above in |ES87, 
INak99l one can see that the C* 2 -character of the fiber £ q is 

Mb)+l.-a(b) ,,-l(b), a(&)+l 



t -l £ f «(6)-/(b)-i f -«(fc) +f /(fc)-fl(b)-l f «(b)+l_ 

1 V-n «(&)-/(fe)-l .-«(&) ,f(fc)-«(fe)-l 

^2 / j ^2 1 T t 2 1 " 



bG7T 3 

By Atiyah-Bott localization formula, the contribution of the fixed point 
Q corresponding toll G Fix„ to the integral /niIb 2 (P 2 ) c 2«(£) * s 

c = ((a - I - 2)si - as 2 ) ((Z - a - 2)si + (a + l)s 2 ) x 
j44 ((a - / - l)si - as 2 ) ((/ - a - l)si + (a + l)s 2 ) X 

((a - j - 2)s 2 - asi) ((? - a - 2)s 2 + (a + l)si) 
/J 3 ((«-!- l)s 2 - flSl ) ((/ - a - l)s 2 + (a + l) Sl ) ' 

where s, = Ci(f;), and a = fl(fc), / = 1(b). 
So we have shown 

7[Hiib«(F2)] 
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Using this formula one can easily evaluate 

/ , c 2 (£) 

./[Hilb^P 2 )] 
,/[Hilb 2 (P 2 )] 

2. Threefolds fibered over curves and proof of TheoremQ] 

Let C be a nonsingular projective curve and X be a nonsingular threefold 
that admits a projective surjective morphism n : X — > C with irreducible 
fibers. By Bertini's theorem general fibers of n are nonsingular. In other 
words, we allow finitely many fibers to be possibly singular or nonreduced. 
We denote by F the class of the general fibers of re, and we fix a polarization 
L on X such that C\(L) is a primitive class, and we let £ = FL 2 . Suppose 
J 7 is a coherent sheaf on X with ch(J 7 ) = (0,rF,j,w), then the Hilbert 
polynomial of J 7 is given by 

(3) P(m) = (r£/2)m 2 + L(j + rFc 1 /2)m + w + + rF(c\ + c 2 )/12, 

where c\ and c 2 are the first and second Chern classes of X. We consider 
A4 = A4(X;P), the moduli space of semistable 2-dimensional coherent 
sheaves with Hilbert polynomial P. Any C-point of M. corresponds to an 
S-equivalence class of a coherent sheaf J 7 which is (at least set theoretically) 
supported on the fiber(s) of re. We can be more specific with the scheme 
theoretic description of the support if J 7 turns out to be stable: 

Lemma 2.1. Suppose that J 7 is a stable sheaf on X with Ci(F) = rF, then the 
support of J 7 is reduced and connected. 

Proof The support of J 7 is connected since T is stable. To see the support 
of T is reduced, denote by S the support of J 7 with the reduced induced 
structure, and let 1$ be the ideal sheaf of S in X. We have F = k[S] for some 
positive integer k because the fibers of X are irreducible. We also have 
that 1$ | s is a torsion invertible sheaf of order at most k. Therefore, since 
Ci(J-") = rF, we see that T and T <g> 1$ have the same Hilbert polynomial. 
Tensoring the short exact sequence — > Jg — > Ox — > 0$ — > by J 7 we get 
the exact sequence 

J 7 <8> I s -> T -> J 7 <8> O s -4 0. 

By the stability of J 7 and J 7 (8) 1$, the first map in the sequence above is either 
an isomorphism or the zero map (see [HL97| [Proposition 1.2.7]). But the 
former is not possible, so we conclude that the second map in the sequence 
above is an isomorphism, and this finishes the proof. □ 

Assumption 2.2. (Condition on the Hilbert polynomial) Let P be the Hilbert 
polynomial of a sheaf J 7 with Ci(J-") = rF. We assume throughout this section 




INVARIANTS OF 2-DIMENSIONAL SHEAVES 



11 



that there are no strictly semistable sheaves with Hilbert polynomial P. f\ M. = 
M. (X; P) is then the moduli space of stable sheaves supported on the fibers of n. 



Assumption 2.3. (Condition on the canonical bundle Kx) We assume throughout 
this section that the restriction ofKx, the canonical bundle ofX, to the fibers of re is 
trivial. This implies that for any sheaf J 7 with c\ (J 7 ) = rF we have T Kx = T 
as T is supported on a fiber of n. Serve duality then implies that 

Ext 3 (J-,J-) ^Hom(J,J) v and Ext 2 (J 7 / J 7 ) = Ext 1 (J 7 / J") v . 



By Assumptions |Z2| and |23| Ext 3 ( T, T) Q = over M(X;P). This en- 
sures that M.(X;P) admits a perfect obstruction theory and hence a 0- 
dimensional virtual cycle [.M] "" (see [ThoOOJ). The corresponding DT in- 
variant is then defined to be the degree of this cycle: 

DT(X;P) = / 1. 

J[MY ir 

Notation. We denote by n M : X x M -4 M and n x : X x M ->■ X 
the natural projections. Let p : M. — > C be the natural morphism which, at 
the level of C-points, sends the sheaf J 7 with support S to 7t(S) G C. The 



morphism p is well-defined by Assumption 2.2 and Lemma 2.1 Define 



S = X n x p M 4XxjW. 

Then the universal sheaf IF ^ is the push forward of a rank r torsion-free 
sheaf G supported on S i.e, F = i$*G. We denote by n' M and n' x the com- 
positions of z'g with tzj\4 and nx respectively. 
Using this notation we have 

Theorem 2.4. f[Tho00|) There exists a perfect deformation-obstruction theory 
over M. given by the following morphism in the derived category: 

E' := (r^Rn M ,{R^omx^M{^^)) y [-1] -> ^m- 

□ 

Example 2.5. (The Enriques Calabi-Yau threefold IIMP06bl ) Let Q = (S x 
E)/Z2 be the Enriques Calabi-Yau threefold obtained from Z2 acting by 
the Enriques involution on the K3 surface S and by —1 on the elliptic curve 
E. The projection to the second factor then gives re : Q — > P 1 which is a 
K3 fibration with four double Enriques fibers T over p\, pi, p^, ^4 G P 1 . Q 
fits into the setting of this section. Since Q is a Calabi-Yau threefold the 
obstruction theory above is symmetric. 



n 

This is the case for example when F is an irreducible class and r = 1. 



4 



F may exist only as a twisted sheaf (see |Tho00 Section 3] for a relevant discussion). 
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If r = 1/2 then M. ( Q; P) is isomorphic to a union of the copies of Hilb" ( T) 
for some n, and hence 

DT(Q;P)=8£ E X(Hilb ;! - ? W(r)). 

hGZ. jeH 2 (X,Z.) 
L-7 = P'(0) 



If r = 1 (recall that P satisfies Assumption 2.2 1 then Ai(Q;P) is nonsin- 
gular at any point of p^ 1 {p) where p ^ p,. In fact, the general fibers of 
p : A4 (Q; P) — > P 1 are isomorphic to (a union of) Hilb" (S) for some n, and 
the fibers over p\, . . . , p± are isomorphic to the moduli space of rank 2 stable 
sheaves on T with Hilbert polynomial P. Since Q is a Calabi-Yau threefold 
we can write 

DT{Q;P) = -2 £ E ^(Hilb ,I+1 - p (°)(S))+4^(A^(T;P) / t/ A1(Q;P) ). 

freZ -yeH 2 (X,Z) 
L-7 = P'(0) 

Alternatively one may use the degeneration and localization techniques 
(as in [MP06b]) to evaluate DT invariants of Q. 



2.1. Smooth fibrations. We suppose that Assumptions 2.2 and 2.3 are both 
satisfied, and furthermore, we assume that the morphism 

(4) 7i : X -> C 

is smooth. Then the fibers of n are nonsingular projective surfaces with 
trivial canonical bundles (by Assu mpti on |2.2| . Suppose that the Hilbert 



polynomial P satisfies Assumption 2.2 The moduli space Ai(X;P) can 
have several connected components: 

Definition 2.6. We call a connected component M c of M. = M{X;P) a 
type I component if jo(.A4 c ) = C, otherwise we call M. c a type II component. 
If M. c is a type II component, then there exists a point p G C such that 
|0(7W C ) = p; if the tangent sheaf of TWc is isomorphic to the tangent sheaf of 
M.{rc^ 1 {p);P) we call M. c an isolated type 2 component. In other words, 
an isolated type II component is isomorphic to the moduli space of rank r sta- 
ble sheaves with Hilbert polynomial P on one of the fibers of n. We usually 
denote a type I component by M.q and an isolated type II component by 
M is0 . 

Mukai in [Muk84 [ proves that the moduli space of stable sheave on K3 
or Abelian surfaces is smooth and irreducible. This immediately implies 
that 

Lemma 2.7. If M. c is a type I or an isolated type II component of M.(X; P) then 
M. c is smooth and irreducible. 

□ 

In what follows we study the restriction of the virtual cycle of M. (given 
by Theorem |2.4| | to the type I and the isolated type II components. We know 
that for any fiber S of n and any sheaf Q supported on S, Extg = 0. 
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This implies that a type I component M.q of M. admits a perfect p-relative 
obstruction theory. The following theorem proven in [ThoOO . HT10 j gives 
the natural p-relative perfect obstruction theory over a type I component 
Mq (in the sense of IBF97I ): 

Theorem 2.8. There is a p-relative obstruction theory over a type I component 
M.q given by the following morphism in the derived category: 

\^Rn' Mo ,(R^om s (G,G)y [-1] L^ o/c - 

□ 

Proposition 2.9. The p-relative obstruction theory over a type I component M. Q 



given by Theorem 2.8 induces a perfect obstruction theory F' — > L^ o over M.$. 



Proof. Composing the morphism in the derived category in Theorem 2.8 
with the second morphism in the exact triangle 

Lk^ L M,/C^P*^c[l], 
we obtain a morphism in the derived category 

A : (r^Rn' Mo ,(RJ^om s (G,G)y [-1] -> p*cv c [l}. 



Now the two exact triangles below induce a morphism g (the left vertical 
map) in the derived category. 

Cone(A)[-l] ► (r^R7t' Mo ^RJfom s (G r G)^ V [-1] > p*w c [l] 

g id 

^M > ^M /C > P*"C[1) 

Define F' := Cone(A)[— 1], then comparing the induced long exact se- 
quences of cohomologies of the exact triangles in the first and second rows 
of diagram (2T I proves that g : Fq —> L^ o defines a perfect deformation- 
obstruction theory for M.q. □ 

We use the following proposition to compare the obstruction theories 
given by Theorems 2.4 and 2.8 on a type I component M.q\ 

Proposition 2.10. There exists the following exact triangle in the derived category 
ofM Q : 

t^Rtt Mo , (i s *RJ?om s (G,G)) -> x^Rn Mo * (R^om XxMo (F,F)) 
-> t^ 2 Rtz Mo * (i Sst R3eom s {G,G® O s {S))) [-1] 
Proof. Applying the functor RJ4?om( — ,G) to the exact triangle 
G ® O s (-S)[l] -> Li* s is*G -> G, 
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we obtain 

RJTom s (G,G) -> RJ^om s (Li^i s *G,G) -> RJtom s (G ® S (-S),G)[-1], 
which by the adjoint property Lig h z's* gives 
(5) 

f SHl RJf ; om s (G / G) -> RJfom Xx M (F,F) -> i s *Rifoffls(G,G®0 s (S))[-l]. 

The result follows by applying the functor Rtzm * and then truncating the 
last sequence above. Note that after applying Rtzmq* to (|5) the terms of the 
resulting exact triangle from left to right are respectively concentrated in 
degrees [0,2], [0,3], and [1,3]. Moreover, the 0th cohomologies of the first 
two terms, and the 3rd cohomologies of the last two terms are isomorphic. 
As a result, after truncation we still get an exact triangle. □ 



2.4 



and Proposition 2.9 produce the virtual cycles [Ado, £*] 



Theorem 

and [.Mo, F'] mr (see IIBF97I ). The relation between these two cycles is given 
by 

Proposition 2.11. Let Adobe a type I component of AA (X; P) and let Obo be the 
locally free sheaf on AAq obtained by restricting Ext 1 , (G, G(S)) to Mo then 

[M ,ET r = c top (Ob ) n [A4 ,F T r . 



Proof By Proposition 2.10 Obo fits into the short exact sequence of the vec- 
tor bundle stacks over AAq (see [BF97|): 

feV^CO -4 /2V^°(E ,V ) -4 Ob . 
Denote by 0ob and E .v the zero section embeddings of AAq. Then this 
gives the Cartesian diagram 



feVfc (Fo" V ) 



Mi 



^h l /h°(E' y ) 

i 



°Ob n 



-^Ob 



>o 



Now if £m is the intrinsic normal cone of AAq, from the diagram above, 
we get (see IIKre99l , IBF97I ) 

[Ai Q ,E'\™ = ti E .,[<l Mo ] = (ioO F .v) ! [£^J = 0^oi l [C Mo ] 

= °f -v o 05 bo [<£ Mo \ = Cf p(Obo) n O^v [€ Mo ] 

= c top (Ob )ri[M ,FZ} vir . 



□ 



For our later application we need to express c ft , p (Obo) in terms of the top 
Chern class of the tangent sheaf of A4o, where Obo and Mo are as in Propo- 
sition 2.10 The component Ado is irreducible and smooth (see Lemma 2.7 1 
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with the relative tangent sheaf T Mo /c = Sxt\, (G,G) (see MHL971 Corol- 
lary 4.5.2] and |Leh98|). From our construction one can see that there exists 
an invertible sheaf Co on C such that 0${S) |m x c x — ^m ° P*£o- 

Lemma 2.12. Suppose that the dimension of the type I component M.q is n and 
Tmq/C/ £o an d Obo are given as above. Then we have the following relation in 
H*(M ,Q): 

ctop(Ob ) = c n {T Mo /c) +£A / Uci(p*A>y. 

7=1 

for some Aj G H n ~i(Mo,Q). 

Proof. Grothendieck-Riemann-Roch formula gives: 

ch(^(-iy£xt' n , M (cms)) 

V 1=0 J ° ' 

= n' Mo , ( ch(G) v • ch(G) • ch(0(S)) • n'i td(X)) 

= ch(£xt n , (G,G)) + t Cl( ( £0)i "W( ch (G) v • ch(G) • n'i td(X)) . 

j=i ]■ \ J 

Note also that ch(£xtL, (G,G(S))) and ch(£ xtL, (G,G)) are clearly poly- 

nomials in c\(p* C$) for i = 0,2. The lemma is proven by an inductive 
argument on n. □ 

The type I component M.q is smooth and irreducible (see Lemma 12.71 , 
so in particular it admits a fundamental class [.Mo]- To find the relation 
between the virtual cycle [.Mo, F'] 1 " 1 ' and [Mo], we note that the obstruction 
sheaf 

h\F^) R 2 7z' Mo ,RJ^om s (G,G), 
is an invertible sheaf on .Mo- In fact the trace map defines an isomorphism 

since by Nakayama lemma it is enough to show that tr gives isomorphism 
on the level of fibers over the closed points of M.q, and (fiberwise) the trace 
map is the Serre duality isomorphism. By Grothendieck-Verdier duality. 

R 2 n> Mo ,Os = (R°n> Mo ^ n , Mg )\ 

By the adjunction formula w n i = tc^Kx <S> 0$(S) and hence, by Assump- 
tion |23l we have 



(6) R 2 n' Mo *O s = P *1C 

for some invertible sheaf K on C. The fiber of /C over a closed point c G C 
can be identified with H°(S,K x \s) where S = n~ l (c). 
Now by [BF97, Proposition 5.6] we have 
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Proposition 2.13. Suppose that M.q is a type I component of Ai(X;P), and /C 
is the invertible sheaf on C defined as above. Then we have 

[M 0/ F -r = Cl (p*/C v )n[Mo]. 

□ 

Remark 2.14. In more technical terms [.Mo] = [Mo,F']™J d , where the latter 
is the reduced virtual class obtained by reducing the obstruction theory F' 
(see IIMPT10llKLT0llMP07l ). ' 

We finish this section by studying what happens when we restrict the 
virtual cycle [M., E'] mr to an isolated type II component M.i so - By Lemma 



2.7 M i so is smooth and irreducible, and is isomorphic to a moduli space of 
stable rank r torsion-free sheaves on a fiber n~ l {p) for some p G C. Let 
[A^fso] be the fundamental class of Aij 



l iso- 



Proposition 2.15. Suppose that Mi so is an isolated type II component of M (X; P), 
and let Ti so denote the tangent sheaf of M. iso . Then 

[M,ET r \ Miso = c top (T iso )n[M iso ]. 

Proof. Let p = p{Mi S0 ) G C. Applying Rti' m . % to the exact triangle (|5} in 



the proof of Proposition 2.10 and taking cohomology we get the following 
exact sequence on M.i so : 

Uom n , (G,G) -¥ £xt\, (G,G) ->• ExtL (F,F) -* SxtL (G,G) ->• 0. 

M iso M iso M iso M iso 

Since M.i so is a type II isolated component, by definition, we obtain the 
isomorphisms of the tangent sheaves 

ExtL (G,G)^£xtL (F,F), 

M iso M iSO 

from which we conclude that the first map in the sequence above is injective 
and hence an isomorphism. Hence, the exact sequence above implies that 

h\E- y ) \ MiS0 =£xt 2 n , (F, F) = £xth (G,G). 

M iso M iso 

But the right hand side is isomorphic to the tangent sheaf Tj so arid hence 
a locally free sheaf. Now the proposition follows from [BF97, Proposition 
5.6]. ' □ 

2.2. Smooth K3 fibrations. In this section we assume that the fibers of the 
smooth fibration Q are K3 surfaces. Let i : S X be the inclusion of 
the fiber of n over a closed point p G C, and suppose that a C-point T of 
A4 (X; P) is supported on S. Then T is the push forward of a stable rank r 
torsion-free sheaf Q on S with C\{Q) = ft and C2((?) = x. Since Ci(S) = 
and Ca(S) =24, formula |3| is simplified to 

P(w) = (r£/2)m 2 + dm + fi 2 /2 -r + 2r. 
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WehaveP'(O) =d = i*L-fi = L-jwherej = ^(F), and P(0) = #(.F) = 
^ 2 /2 - t + 2r with £ 2 /2 - r = ch 3 ( J 7 ). Let 

(7) z;= (r /J 6 /J 6 2 /2-T + r) 

be the corresponding Mukai vector and Ai(S;v) be the moduli space of 
semistable sheaves on S with Mukai vector v. Then AA(S;P), the moduli 
space of stable coherent sheaves on S with Hilbert polynomial P (with re- 
spect to i*L), is a finite disjoint union of the moduli spaces Ai (S; v) over all 
possible Mukai vectors giving rise to P. By our assumption on the Hilbert 
polynomial we conclude that each Mukai vector v is primitive and hence, 
A4 (S; v) is smooth of dimension 

2- J v ■ v v = 2rr - (r - l)^ 2 - 2(r 2 - 1) 

= 2 + 2r 2 + ^ 2 -2rP(0). 

The geometry of the moduli space M.[S;v) has been thoroughly studied 
IIKYOOi IHL951 IMuk84l IKY11I . The following result has been proven in 
IIMuk84H and HHL97T Section 6] : 

Proposition 2.16. Let S be a K3 surface and v a -primitive Mukai vector as in 
Q. Then M. (S; v) is deformation invariant to Hilb" (S), the Hilbert scheme ofn 
points in S, where 

n = rr- (r - l)/S 2 /2 - r 2 + 1. 

In particular, 

X {M{S;v))=x^\h n {S;v)). 

□ 

We denote by .A/f^ch^) the component of .M(X;P) corresponding to 
the Chern character vector 

(8) ch, = (0,^,7 = i*p,ca := /3 2 /2 — t) G ©f =0 H 2 '(X,Q) 
assigned to the primitive Mukai vector Q. 

Remark 2.17. For our later use, we need to extend the construction of 
M.(X;c\\ v ) to the case where the K3-fibration n : X — > C is possibly not 
projective. We consider the case where there are finitely many K3 fibers 
S of tc for which i*L is only a quasi-polarization (see [MP07J). Further- 
more, we assume that there is a finite open affine cover liy of C such that 
is projective. In this case for the compactly supported Chern char- 
acter vector ch£ s assigned to the primitive Mukai vector ^ the moduli 
spaces Ai ( IT,-; ch^f ) can be constructed using the fiberwise polarizations. 
Ai(Uj;ch c v s ) and Ai(Uj;ch c v s ) are canonically isomorphic over the overlaps 
Ui Pi Uj, and so they can be patched together to give a proper scheme M. (X; chj,) 
over C. Two constructions above obviously give the same result when 
X is projective over C. The perfect obstruction theories and the virtual 
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cycles over M.(Ui;cb.™) constructed in the last section are also glued to- 
gether to give the corresponding virtual cycles over A4(X;ch v ); we define 
DT(X; ch v ) as before, using the properness of M. (X; ch v ). 

Following IIMP07II , let 

V : R 2 7r,(Z) -> C 

be the rank 22 local system determined by the K3-fibration n. Let T-L v de- 
note the 7r-relative moduli space of Hodge structures as in [MP07, Section 
1.4] There exists a section map cr : C — > 7i v which is determined by the 
Hodge structures of the fibers of n: 

(9) a{p) = H°{X,K S ) eU v K 

For any 7 G H 2 (X,Z) 7T and h G Z let 

V p (h,<y) = {0^peV p \p 2 = 2h- 2,u(fi) = 7}, 

and B p (h, 7) C V p (h, 7) be the subset containing f> G V p (h, 7) where f> is a 
(1, 1 ) class on S (the fiber over p). Then B p (/z, 7) is finite by Proposition 1 in 
[MP07|. B(/i / 7) = Up B(/z,7) C V can be decomposed into 

where the first component defines a finite local system e : Bj(/i,7) — > C, 
and the second component is an isolated set. Let 7W e be the connected 
component of M. (X; chj,) corresponding to the local system e, and let 



DT X;e = / 1 

J[M e ,E'Y" 

be the contribution of this component to DT(X; chj,). Not e that A4 £ is a 



type I component of .A/f^chu) in the sense of Definition 2.6 and [.A4 e/ E*] 1 '"' 
is the restriction of the virtual cycle given by Theorem |2.4| to this compo- 
nent. Denote by \Ai €l F*] mr the virtual cycle over this component given by 



Proposition ^91 By Propositions 2.11 and 2.13 we have 



[M £r E'f = c top (Ob £ ) n [M e ,F' e r, 
(10) [M e ,FZf r = c t (p*lC v )n[Me], 

where [M. e \ is the fundamental class of M. e , and Ob e is the locally free 



sheaf on A4 £ defined as in Proposition 2.11 

On the other hand, if a G Bn{h, 7) supported on the fiber S is a result of 
a transversal intersection^of cr(C) with a Noether-Lefschetz divisor, then the 
corresponding connected component A 4 a of A4(X;ch v ) is an isolated type 



II component in the sense of Definition 2.6 and is isomorphic to Ai(S;v) 



5 In IMP07I Section 1.4] this is denoted by M v . 
^i.e. with the intersection multiplicity 1 
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Denoting by Tm x and [M. a ] the tangent sheaf and the fundamental class of 



Ai a , by Proposition 2.15 we have 



(11) [M a ,ET r = c top (T M J n [M a ], 

where [M. a ,E'] mr is the restriction of the virtual cycle given by Theorem 



Z4|to M a . We define 

DT(X;u) = / 1 
J[Mu,E'\™ 

the contribution of DT(X; chj,) to this component. 

For any integer JiGZ the Noether-Lefschetz number was defined 

in [MP07| by intersecting cr(C) with the 7r-relative Noether-Lefschetz di- 
visor in 7l v associated to h and 7. Informally NL^ 7 is the number of the 
fibers S of n for which there exists a (1,1) class f> £ H 2 (S,Z) such that 

B 2 = 2h-2 and i.jB = 7. 

It is proven in IIMP07I that NL? vanishes if h > 1 + d 2 /2t and 

(12) NL£ 7 = NL% +d+i/2 , 7+F . L . 

The following theorem which is the main result of this section is a sheaf 
theoretic version of [MP07, Theorem 1]: 

Theorem 2.18. For a smooth K3 fibration re : X — > C and the Chern Character 
vector ch v as in ([8} we have 

DT{X;cK) = E xmb h - r "- r \s)) ■ NL%. 

heZ 

-k^^Hilb 1 -^^)), 

where 5q i7 is the Kronecker delta function, and k is the degree of the invertible sheaf 
K. onC given in 

Proof. The proof follows the same ideas as the proof of Theorem 1 in [MP07] . 
We compare the contributions of Bj and Bjj to DT(X; ch v ) and the Noether- 
Lefschetz numbers. Suppose that e is a local system giving rise to Bj(h, 7) 
as above. We can then write 



7 



DT{X;e) = [ c top {Ob e ) • Cl (p*/C v ) 

J [Me] 



c top{T M[s . v) ) • / ci(p*/C v ) 

[M{S;v)\ 1 j 7b,(^7) 



^(HUb"-™-' 2 (S))- / Cl (p*/C v ), 
where the first equality is because of pO), the second equality holds by 



Lemma 2.12 and the last equality is due to Proposition 2.16 
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By virtue of it is shown in the proof of Theorem 1 in [MP07J (pp 
22-23) that 

/ C!(/C V ) 
■JB,(h, 7 ) 

gives the contribution of Bi(h, 7) to NLfi . 

Next, suppose that oc S Bu(h,y). Using the deformation invariance of 
DT invariants and the intersection numbers, we may assume that the cor- 
responding component M. K is an isolated type II component after possibly 
a small analytic perturbation of the section a (which locally turns a multi- 
plicity n intersection into n transversal intersections). Once this is done, the 
contribution of oc to NL? is exactly 1, and moreover we can use (111 and 
Proposition 2.16 to deduce 

DT(X;oc) = / c top (T Mx ) = x(Hilb ,! — 2 (S)). 

The proof of theorem is completed by adding the correction term involv- 
ing the Kronecker delta function to take into account the contributions of 
B = to DT(X; ch*). □ 

Corollary 2.19. For a smooth projective K3 fibration re : X — > C and the Hilbert 
polynomial P satisfying Assumption\2.2\we have 



DT(X;P) = £ £ X (Hi[b r2+h - rP{0 \s)) ■ NL^ 

;Z 7eH 2 (x) 

L-7 = P'(0) 

V(0)-k^(Hilb r2+1 -'- p (°)(S)). 



□ 



Now define 



m,d = E NL lr 



7 e H 2 (X) 
L-7 = rf 

Then the formula in Corollary |2.19 for d 7^ can be rewritten as 

DT(X; (r£/2)m 2 + dm + c)=J2 ^HUb^^S)) • NL% d . 

fceZ 

By p2| we get the following symmetry among the DT invariants of X: 

DT(X; (r£/2)m 2 + dm + c) = DT(X; (r£/2)m 2 + (d + £)m + d), 

where d = c — {2d + £)/ 2r, provided that d is an integer, and both Hilbert 
polynomials satisfy Assumption |2.2[ This is true for example in the case 
r = 1. 
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2.3. Nodal K3 fibrations. In this section we aim to extend Theorem 12.181 
to the case where the K3 fibration has finitely many ordinary double point 
singularities (ODP). Using the deformation invariance, finding the DT in- 
variants of X, when the singularities of the fibers are of more general type 
of rational double points (RDP), may be reduced to this case. We assume 
throughout this section that r = 1, so all the pure sheaves are automatically 
Gieseker stable. 

Let Si, . . . , Sfc G X be the singular points of the fibers of X, and assume 
that there are k! singular fibers in X over C\, . . . , G C where k! is a non- 
negative integer. Moreover , we assume that s, is an ordinary double point 
(nodal) singularity for any i, and the other fibers of X are smooth K3 sur- 
faces. If k! is even, define Co = c\ and if k! is odd, define Co to be an arbitrary 
point of C distinct from c\, . . ., cy. Define e : C — > C to be the double cover 
of C branched over the points cq, . . . , cy. It can be seen that e* (X) is a three- 
fold with the conifold singularities. Denote by X its small resolution with 
the exceptional nonsingular rational curves e\, . . . , e^, and let n : X — > C 
be the induced morphism. The normal bundle of e ; in X is isomorphic to 
O p i(-1) © Opi(-l) IAti58l . Moreover, let e t : Q -> C be a double cover 
of C branched at k + 2{k/2} generic points of C when t ^ and Q = C. 
Define X t = e*(X). Our plan is to relate the DT invariants of X and X t 
which differ by the conifold transitions. As in GW theory MLY04I , BLR01H 
this can be done using the degeneration techniques. It is possible that X is 



no longer projective in which case we use the modifications of Remark 2.17 
to define the DT invariants. See Appendix [A] for a review of the degenera 
tion techniques in DT theory. 

We use the degenerations of the threefolds X and Xf to respectively 

X':=Y (J IJ Pi and X" := Y (J IJP 2 

D u ...,D k (=1 D lr ...,D k !=1 

where Y is the threefold obtained by blowing up e* (X) at 

e~ 1 (si) / ... / e~ 1 (s Jt ) 
with the exceptional divisors D\ TJ • • • TJ (each isomorphic to P 1 x P 1 ), 

Pi = p(o pl ©e> pl (i) 2 ), 

and P2 is a quadric in P 4 . The first degeneration is the degeneration to the 
normal cone [Ful98J in which D, C Y is attached to the divisor at infinity 
Hi = P(Opi (l) 2 ) in the z'-th copy of Pi. The second degeneration is called 
the semistable reduction of a conifold degeneration 1LY04| in which D, C 
is attached to a smooth hyperplane section H2 in the z'-th copy of P2. We 
denote by X\ — > A 1 and X% ~^ ^A 1 the total spaces of the first and second 
degenerations above. In other words, X\ (respectively, X2) is a nonsingular 
4-fold with the central fiber isomorphic to X' (respectively X"). Let X ; — > £ 
be the corresponding stack of expanded degenerations (see Appendix [A] . 
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Let {^•}|L 1 be an open affine covering of C, where 
V t = C- {ci,.. .,Ci,...,c k i}. 

Define the invertible sheaves L t and L on X t and X as follows: 

_ k 
L t = e* (L), and L = e* (L) - £ D; 

2=1 

where e : X — > X is the natural morphism, and D, is a divisor on X with 
e, ■ Dj = —2 and e • D, = for any other curve e on IT, = ft _1 (T^). L t defines 
a polarization on Xf and L defines a polarization on IT, for each j = 1, . . . , k 1 . 
We also denote by F the class of fibers in the fibrations X t — > Q, X — > C, 
and Y — >■ C. We denote by .M (X; P) the moduli space of 2-dimensional 
sheaves J 7 supported on the fibers of X — > C such that c\ (J 7 ) = F, with 
Hilbert polynomial P with respect to L. Note that L is a polarization on 
Ui's and A4 (X; P) is constructed by gluing the moduli spaces M. (IT,; P) as 
in Remark |2.17| 

Fix an invertible sheaf C\ on X\ whose restriction to a general fiber is L 
and to Y is 

where b : Y — > X is the natural map. The invertible sheaf V defines a 
polarization on Y and we let A4(Y /D\, . . . , D&P') be the relative moduli 
space of stable sheaves T with Hilbert polynomial P' (with respect to V) as 
in Definition IA.21 

Similarly fix a relatively ample invertible sheaf £2 on Xi whose restric- 
tion to a general fiber is L t and to Y is V '. Let M. (£,-/ £; T 5 ;) be the relative 
moduli space of stable 2-dimensional coherent sheaves with Hilbert poly- 
nomial Vi with respect to (see Definition A.2| . 



Note that the general fibers of X\ are not necessarily projective and hence, 
the moduli space is constructed b y glu ing the relative moduli spaces over 



the open cover {Uj} as in Remark 2.17 



Lemma 2.20. i) Suppose Vi is such that for any C-point T of M(Xj/ f £;Vi) 
we have C\{F) = F. Then A4(Xi/<£;Vi) admits a perfect obstruction theory 
relative to £ and a virtual cycle of dimension 1. 
ii) Suppose P' is such that for any C-point T of M.(Y /Di, . . . ,Dj.;P') we 
have C\ (J 7 ) = F. Then M. (Y/Di, . . . , D^; P') admits a perfect deformation- 
obstruction theory and a virtual cycle of dimension 0. 

Hi) Suppose P' is such that for any C-point T of M. (Y /D\, . . . , Dj.; P') we have 

c x (T) = F-D h D ig . Then M {Y/D lf ...,D k ;P') admits a perfect 

deformation-obstruction theory with virtual cycle equal to 0. 

Proof, i) For any T as in the proposition we have Ext 3 ( T , J r )o = by Serre 
duality and the stability of T and noting that T = T <g) o^x /ik 1 where 
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cox /A 1 * s the relative dualizing sheaf. Now the claim follows from the ex- 



planation after Definition A.2 



ii) As in part i) it suffices to show that Ext 3 ( J 7 , J-")o = for the C-points J 7 

in M (Y/Di, . . . , D k ; P'). We know that K Y = D t H h D k and F • D, = 0. 

Since ci (J 7 ) = F by assumption, Serre duality and the stability of T imply 
that 

Ext 3 (J 7 , J 7 ) S Hom(J 7 / J") v S C, 

and hence Ext 3 (J 7 / J 7 ) = 0. 

iii) For simplicity we assume that C\ (J 7 ) = F — Di . As in previous parts 
we prove Ext 3 (J 7 , J 7 ) = 0. By Serre duality 

Ext^J 7 ,/ 7 ) =Hom(J- / J-(D 1 )). 

But T is rank 1 supported on a smooth K3 surface by our assumption, and 
hence T is isomorphic to an ideal sheaf of points X twisted by a an invert- 
ible sheaf L. Therefore, 

HamO^-FCDi)) ^ Hom(X,X(^)), 

where ei = P 1 is given as the intersection of D\ with the proper transform 
of the fiber containing the curve e\. Since e\ is a — 2-curve on a K3 surface, 
we know that H (O(ii)) = C which implies that Hom(J, l(e t )) = C as 
required. 

To prove the vanishing of the virtual cycle, we show that the virtual di- 
mension is negative. By the Hirzebruch-Riemann-Roch calculation: 

ext° (J 7 , J 7 ) - ext 1 (J 7 , J 7 ) + ext 2 (J 7 , J 7 ) - ext 3 (J 7 , J 7 ) 

= /'((F-D 1 ) + ^ + n)(-(F-D 1 )+ J S-n)(l-^A + ...) 

= (F-D 1 ) 2 -D 1 /2 = l, 
from which we get 

ext 1 (J 7 , J 7 ) - ext 2 (J 7 , J 7 ) = -1. 

□ 

Lemma 2.21. For i = 1,2 let T{ be a pure 2-dimensional sheaf on P, with 
C\{F) = Hj, and let Q, be the Hilbert polynomial of J 7 } with respect to any po- 
larization. The relative moduli space of stable sheaves A^(P t -/Hj; Qi) admits a 
perfect deformation-obstruction theory. 

Proof. The canonical bundle of P, is Kp j = — 3H,-. So for any C-point of 
Ai given by a pure 2 dimensional sheaf T on P„ we have Ext 3 (J 7 , J 7 ) 9* 
Horn (J 7 , J 7 , ®Kp ; ) = by the stability of J 7 and Serre duality as before. □ 

Now we are ready to state the conifold transition formula for the DT 
invariants: 
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Proposition 2.22. (Conifold Transition Formula) Suppose that P e Q[m] is a de- 
gree 2 polynomial with the leading coefficient equal to FL 2 /2. The DT invariants 
ofX t and X are related by the following formula 

DT(X t ;P t = P) = DT(X,P = P) 

where P t is the Hilbert polynomial with respect to L t/ and P is the Hilbert polyno- 
mial with respect to L. 



Proof. By Lemma 2.20 M.{3L\/<L,V\) admits a perfect obstruction theory. 



Applying the degeneration formula ( [17} which is followed from the nat- 
urality of the virtual cy cle \M .{3L\/ '£;Vi)] mr (see Appendix |a|), part ii) of 



Lemma 



2.20 



and Lemma 



2.21 



. we can express DT(X;P = P)m terms of the 
relative DT invariants of Y and Pi . There are two possibilities for a C-point 
T on the central fiber of M.{3L\/ <t,Vi). Either T is completely supported 
on Y (and possibly its degenerations) or there are some i\, ■ ■ ■ , ig such that 

Ci(J^|r) = F - D h D ig , and c x (T\ i} ) = H,-. where pj is the z' ; -th 

copy of Pi. Only the former case contributes because of the vanishing of 
the virtual cycle proven in part iii) of Lemma 2.20 Therefore, 



DT(X; P = P) = DT(Y/D lf . . . , D k ; P' = P). 

Similarly, using the degeneration formula ( fl7| l followed from the natu- 
rality of the virtual cycle [M (X 2 /€; V 2 )] vir , we can express DT(X f ; P t = P) 
in terms of the relative DT invariants of Y and P2. Again by the same argu- 
ment as in the last paragraph by distinguishing two similar cases and using 

we get 

DT(X t ; P t = P) = DT(Y/D 1 , D, f ; P' = P). 



Lemmas 2.20 and 2.21 



Now the lemma follows from the last two identities. 



□ 



Now we choose k' generic fibers Si, ... , Sjf of X — > C. By our assump- 
tion Sf is a K3 surface. Let M. (X/S\, . . . , S^; P) be the relative moduli space 
of stable sheaves J 7 on X with c\ (J 7 ) = F and Hilbert polynomial P (with 
respect to L) as in Definit ion |A.2 By our conditions, one can see simi- 
lar to the proof of Lemma 2.20| that M.(X/ Si, ... , S; C /;P) admits a perfect 
deformation-obstruction theory and virtual class of dimension zero. 

Let X, = S, x P 1 . Then X, is a smooth K3-fibration over P . Once more, 
we denote the class of the fibers by F. Let Ai (X, / Sf, Q) be the relative mod- 
uli space of stable sheaves J 7 on X with c\ (J 7 ) = F and Hilbert polynomial 
Q with respect to any polarization. 

Lemma 2.23. DT(X i /S l ;Q) =0. 

Proof. The invertible sheaf K associated to the fibration X, as in (|6]l is eas- 
ily seen to be trivial in this case, and hence DT(X„ Q) = 0. Next, by the 
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degeneration formula (T7\ and the irreducibility of the class F, we get 

2DT(X i /S i ;Q) = DT(X i ;Q) 
from which the result follows. □ 
Lemma 2.24. DT(X/S lr . . . , S k ,;P) = DT(X;P). 

Proof. We use the degeneration formula ([T7]l for the degeneration of X into 

X U II*- 

Si,...,S r i 

The irreducibility of the class F, and the vanishing of DT(X,/ S,, Q) from 
Lemma [2.23 proves the claim. □ 



We use Lemma 2.24 to relate the DT invariants of X to X t . To achieve this 



we use the degeneration of X t obtained by degenerating its base C to two 
copies of C by attaching two copies of X along the generic fibers Si, ... , S^. 
The degeneration formula ( |T7| | then implies that 

DT(Xf,P t ) = 2DT{X/S X ,...,S K ;P) 



where the Hilbert polynomial Pf as in Proposition |2.22 is computed with 
respect to the polarization L t , and P is computed with respect to the polar- 
ization L. This together with Lemma 2.24 and Proposition 2.22 proves 



Theorem 2.25. Suppose that P G Q[m] is a degree 2 polynomial with the leading 
coefficient equal to FL 2 /2. Then DP(X;P) = 1/2DT(X;P = P). 

□ 

2.4. Modularity of the DT invariants. In this section we assume the set 



up and notation of Section 2.3 Theorems 2.18 together with modularity 
property of the generating series of the invariants ^(Hilb"(S)) MGot90l and 
Nfi MMP071 IBorM |Bor99[ IKM90I suggest the modularity of the generating 
series of DT(X;P). To explore this we first put the DT invariants into a 
generating function. We assume that r = 1 throughout this section. Let 

P(m) = (£/2)m 2 + dm + c 

with c,d G Z. Since r = 1 then P(m) satisfies Assumption 
for any C-point T of M(X;P) we have ch^J") = F, L • ch 2 ( 
ch3(J-") = c — 2. Our convention is that when X is a smooth K3 fibration 
(i.e. when k = 0) then X is taken to be the disjoint union of two copies of 
X. Define the generating series 

Md 2 /2£ £ Dr(x . (£/ 2 ) m 2 



and then 

= d, and 



Z d {X,q) = q 



dm + c)q 



cez 



and let 



e-i 



Z(X,q) = "£Z d {X;q)v d GC[[q 

d=0 



■C[Z/£Z]. 
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Following IIMP07I . for NLfa = £ NL£ r define 

7 6 H 2 (X) 
L-7 = d 

*f (?) = <? 1+d2/2f E ^ 

and set 

o-^) = £>f0?> d e C[[^ 1/2 ^] ®C[Z/4fiZ] 



d=0 



noting that NL£ rf = if ft > 1 + d 2 /2£. Then <& n {q) is a vector valued 
modular form of weight 21/2 IIMP071 IBor98l IBor99HKM90l . By HGot90H we 
know 

E^(Hiib"(s))-^ = n fl riw' 



n>0 n>\ 



so E /C(Hilb n (S)) • g" -1 = t](qy 2i , where 77(9) is the Dedekind Eta func- 

)3>0 

tion which is a modular form of weight 1/2. 

Combining these formulas with the results of the Sections 2.2 and 2.3 we 
can express the generating function of our DT invariants in terms of the 
product of two series with modular properties: 

(13) Z(X,q)- 2j ^ )M . 

The formula in Theorem [I] is a special case of p3| . 

Example 2.26. (Lefschetz pencil of quartics [MP07J) 

Let n : X 4 2 C P 3 x P 1 — > P 1 be a general hypersurface of type (4,2). X 4/2 
is then a smooth Calabi-Yau 3-fold, and 71 is a family of K3 surfaces with 
216 nodal fibers over P 1 . The generating series Z(X 4/2 , q) is then given by 
( |T3"| | for which O 71 is explicitly evaluated in [MP07] in terms of the modular 
form 0: 

© = -1 + IO89 + 320^ + 5016^z H 

appeared for the first time in IIKKRS04HMP07II . 

On the other hand, since X 4/2 is a Calabi-Yau 3-fold, one can also evaluate 
the DT invariants by means of the weighted Euler characteristic: 

DT(X;P)=x(M)=x(M,v M )f\ 

Let C\, . . . , C216 € C be such that jt -1 (c;)'s are the singular fibers of n. We 
denote by S the general smooth fiber and So the nodal fiber. We have M.q := 



7 More generally, if A C B is a locally closed subspace then |Beh09| defines f>) — 

X(A,V B \ A ). 
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p^ 1 (c,) = M. (So; P), the moduli spaces of rank 1 torsion-free sheaves on the 
nodal K3 surface So with Hilbert polynomial P. 

216 

DT(X;P) = x(M - {p- 1 ^) p-Hc 216 )},M) + £ l x(P~ 1 (*),M) 

216 

= X(M - {p-\ Cl ) p-\c 21() )}) + Y J x{p-\c i ),M) 

i=l 

= " 214 E E x(Hilb ,I+2 - p (°)(S)) +216x(M ,M). 

ftez t e H 2 (X) 

L-7 = P'(0) 

From this one can express the generating function of the weighted Euler 
characteristic of A4 (So, P) in terms of the modular forms of |13| . 

3. Proof of Theorem[2] 

In this section we study the generalized DT invariants of the Gieseker 
semistable 2-dimensional sheaves of local P 2 . When we mention (semi)- 
stability of sheaves, unless otherwise is specified, we always mean Gieseker 
(semi)stability. 

Let X be the total space of 0{— 3) over P 2 . Then X is a quasiprojective 
Calabi-Yau threefold, called local P 2 . Let L be the pullback of 0(1) from 
P 2 , and let S = P 2 C X denote the zero section. We identify the compactly 
supported cohomology groups of X with the cohomology groups of P 2 : 

H C * S (X,Q) S H*" 2 (P 2 ,Q). 

Using this identification, let H E H C 2 S (X,Q), h £ H% (X,Q), pt S H C 6 S (X,Q) 
be respectively the classes of S, a line and a point on S. The Hilbert polyno- 
mial (with respect to L) of a 2-dimensional compactly supported coherent 
sheaf J 7 on X with the compactly supported Chern character 

ch cs (J-) = (0,kH,(3k/2 + a)h,(3k/2 + 3a/2 + b)pt) e ^ =0 H^ S {X,Q) 
is given by 

„. , k 2 3k + 2a 3k + 2a + 2b 

Pirn) = -m H m H . 

y i 2 2 2 

Any such T is set theoretically supported on S. Moreover, we have 

Lemma 3.1. If J 7 as above is semistable then T is scheme theoretically supported 
on S and hence 

M{X;P) S 7W(P 2 ;P), 
the moduli space of rank k semistable sheaves on P 2 with Hilbert polynomial P. 

Proof. The ideal sheaf of S in X is isomorphic to L 3 , hence we get the exact 
sequence 

J r ®L 3 —>J r —> J 7 |s —> 0. 
Since T is semistable, the first morphism in the sequence above is neces- 
sarily zero and hence T = J 7 ^. □ 
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Note that for any stable torsion-free sheaf J- on P we have Ext 2 (J 7 , J 7 ) = 
by Serre duality and the negativity of K P 2 = 0(—3). Therefore, if P(m) is 
such that there are no strictly semistable sheaves on P 2 with Hilbert poly- 
nomial P(m), then the moduli space M = M(X;P) = M(F 2 ;P) is un- 
obstructed and smooth of dimension 

dimM = l-xiJ 7 ,^) = ~2kb + a 2 - k 2 + 1, 

where as a sheaf on P 2 

ch (J r )=k, ch 1 (J r )=a-h, ch 2 ( J 7 ) = b ■ pt. 

In this case the Behrend's function is determined by i/» = (— \) dimM 
HBeh09l . and hence 

DT(X;P) = {-l) dimM x(M). 

The generating function for the Euler characteristic of the moduli space of 
stable torsion-free sheaves on P 2 is known for k = 1, 2, 3, by the results of 
IIKoo09l|Kly91tlGot901lManT0ll VW94l and they all have modular properties. 
Here is the summary of these results: 

(1) k = 1. By tensoring with 0(— a) we can assume that a = 0. So 
then M{X;P) S Hilb^P 2 ), the Hilbert scheme of -b points on 
P 2 , which is smooth of dimension —2b. Note that in this case there 
are no strictly semistable sheaves on P 2 with Hilbert polynomial 
m 2 + 3m/2 + b + 1, so by IIGot90l 

(14) £DT(X; m 2 /2 + 3m/2 + b + \)q~ h = \\ n nVr 

b n>0 " > 

(2) k = 2. By tensoring with 0(\_—a/2\) we can assume that either 
a = 0orfl = l. Iffl = l then there are no strictly semistable sheaves 
with the corresponding Hilbert polynomial m 2 + Am + 7/2 + b and 
hence M. (X; m 2 + 4m + 7/2 + b) is smooth of dimension —4b — 2 so 
by IIKoo09l Corollary 4.2] 

£ DT(X;m 2 + 4m + 7/2 + &)< ? 1/2 - b 

b e (1/2)Z 

= #(.M (X; m 2 + 4m + 7/2 + &)) 

-i oo oo .mn 

n»>o(i-? B ) 6 ii„rii-? w+B - r 

When a = 0, there are strictly semistable sheaves with Hilbert poly- 
nomial m 2 + 3m + 2 + b only when b e 2Z. If b ^ 2Z then X is 
smooth of dimension — 4b — 3. So for b = 2b' + 1 

DT(X;m 2 + 3m + 3 + 2b') = -x(M(F 2 ;m 2 + 3m + 3 + 2b 1 )). 

We will study the case b G 2Z in more detail in what follows in this 
section. 
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(3) k = 3. We can assume again that a = 0, 1 or 2. In the latter two 
cases M. has no strictly semistable sheaves and there is a modular 
formula for the generating function DT(X; P) in terms of the gener- 
ating function of the Euler characteristics of Ai (P 2 ; P) computed in 
HKoo09l Section 4.3]. 

In the following we compute DT(X;P) in the presence of semistable 
sheaves when k = 2. By the discussion above, we only need to consider 
a = and b 6 2Z. Let P{m) = m 2 + 3m + & be the corresponding Hilbert 
polynomial. We use the moduli space of stable pairs in the sense of | |JS11[ 
(see Definition |0.1| . 

For n S> let V„ = V n (X;P) be the moduli space of stable pairs (T/S) 
where J 7 is a semistable sheaf of rank 2 with Hilbert polynomial m 2 + 3m + 
b, and s is a nonzero section of J-"(n). The stability of pairs (see Definition 
|0.1| further requires that if Q ^ is a proper subsheaf of T , such that s 
factors through Q(n), then the Hilbert polynomial of Q is strictly less than 
the Hilbert polynomial of T . By [JSfTJ V n admits a symmetric perfect ob- 
struction theory. Let PI n = PI„(X; /3) be the corresponding pair invariants. 
Note that, even though X is not proper, V n is proper (as all the semistable 
sheaves are supported on P 2 = S C X) so PI„ is well defined. Alterna- 
tively, PI n = x\Vn,Vv n ). 

Lemma 3.2. DT(X;P) = DT(X;P/2) 2 • P(n)/8 - PI„(X;P) / P(n). 

Proof. This is a direct corollary of the wall-crossing formula IJS11, 5.17] by 
noting two facts. Firstly, the only decomposable semistable sheaves with 
Hilbert polynomial P are of the form X% % © Iz 2 where %z x and Zz 2 are 
the push forwards to X of ideal sheaves of the 0-dimensional subschemes 
Zi, Z 2 C P 2 of length -b/2. Secondly, the Euler form ^(X Zl ,X Z2 ) =0. □ 

Remark 3.3. Note the polynomial in the right hand side ofLemma \3.2\ is a rational 
number independent ofn^O. 

There is a natural morphism V n — > M. that sends a stable pair (T,s) 
to the S-equivalence class of T . Note that M. is singular at a point corre- 
sponding to a strictly semistable sheaf. However, we have 

Proposition 3.4. V n {X;P) is a smooth scheme of dimension P(n) — 4b — 4. 

Proof. We denote by I' the 2-term complex 0{—n) > T corresponding 

to a stable pair {T ,s). By the stability of pairs T has to be a semistable 
sheaf and hence Lemma [3~T] implies that 

V n (X;P)^V n (F 2 ;P), 

the moduli space of the stable pairs on P 2 . The Zariski tangent space and 
the obstruction space at a C-point (f,s) € V„ are then identified with 
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Hom P 2(l' ; J) and Extp 2 (P , J 7 ) respectively. Consider the following nat- 
ural exact sequence: 



Extp 2 



Hom P 2(I* / J") 
{J 7 , J 7 ) 



-> Hom p2 {J r ,J r ) -> Hom p2 (0(-n), J 7 ) 
-> Ext p2 (0(-n), J") -> Extp 2 (I% J") 

Since n > 0, we have Extp 2 (0(-n) / T) S H'(P 2 , .F(n)) = for z > 0. 

We also know that Extp 2 ( J 7 , J 7 ) = by Serre duality and the semistabil- 
ity of J 7 . So the exact sequence above firstly implies that Ext 1 (I*, J 7 ) = 
which means that V n is unobstructed and hence smooth, and secondly 

dimHom(r / F)=^ P 2(J-(n))-^p 2 (J- / J-)=P(n)-4fc-4. 

□ 



By Proposition 3.4 and noting that P(n) G 2Z, we have 



Corollary 3.5. PI n (X;P) = X {V n (X;P)) = x{V n {W 2 ;P)). 

In the following we will find ^(^(P 2 ; P)) using toric techniques. Ac- 
cording to ||Koo09| , a torsion-free T = C* 2 -equivariant sheaf T on P 2 cor- 
responds to three compatible c-families, P\, Pi, P3 one for each of the stan- 
dard T-invariant open subsets U2, U3 of P 2 . For any element m of the 
character group of T identified with Z 2 , Fj(?n) = T(Ui, T) m , the eigenspace 
corresponding to m in the space of sections of T on LI,-. In this way a T- 
equivariant rank 1 torsion-free sheaf X on P 2 is determined by three inte- 
gers u, v, w and three 2d partitions tz\, n' 2 , n'y See the diagrams below indi- 
cating the corresponding (7-families l\, h, I3 over U\, U2, U3 respectively: 



(u,v) (v,zv) (w,u 

For any j = 1,2,3 we have h(m) = if m is below the horizontal axis, on 
the left of the vertical axis, or inside the partition iCj. Otherwise, Ij(m) 



C. 



In terms of these data we have 



chx(X) 
ch 2 (X) 



v — w, 



V + wf/2- #77^ - #TZ' 2 - #Tl' 3 . 



Similarly, an indecomposable T-equivariant rank 2 torsion-free sheaf T on 
P 2 up to a T-isomorphism is determined by an integer A, positive inte- 
gers Ai, A2, A3, three distinct 1-dimensional subspaces p,q,r C C 2 , and six 
2d partitions nj, nj for j = 1, 2, 3. See the diagrams below indicating the 
corresponding cr-families P\, Pj, F3 on respectively U\, U2, U3 and : 
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it— i> 



n\ 



A 2 



T 



7rt 



A 3 



T 



7T 1 I- 



7TS 



1 



(0,0) (0,A) (A,0) 

The points indicated by • have the coordinates (0,0), (0, A), (A,0), re- 
spectively. The partitions nj, nj are respectively placed at the points (indi- 
cated by o) with the coordinates 



Ai 



T 



(0, A 2 ), (A lr 0), (0, A + A3), (A 2 , A), {A, Ax), (A + A 3 , 0). 



We denote by Sj, Sj the vertical and the horizontal strips made by two ver- 
tical and two horizontal lines in each diagram. We also denote by R\, R 2 , R3 
the areas located above the horizontal strip and to the right of the vertical 
strip. For any ;' = 1, 2, 3 we then have 

(Fl) Fj(m) = if either m G nj fl nj, m G Sj D Sj, m G Sj n n 1 -, m is on 

the left of the strip Sj, or m is below the strip Sj. 
(F2) Fj(m) = C 2 if m is in Rj - nj - nj. 

(F3) Fi(m) = p if m G S\ — n\ or m belongs to a connected component 
of n\ — n\ adjacent to a member of S\ — n\; and F\(m) = q if m G 
S\ — n\, or m belongs to a connected component of n\ — n\ adjacent 
to a member of m G S\ — n\. 

(F4) F 2 (m) = q if m G S\ — n\ox m belongs to a connected component 
of n\ — n\ adjacent to a member of m G S\ — n\; and F 2 (m) = r 
if m G S\ — n\ ox m belongs to a connected component of n\ — n\ 
adjacent to a member of m G S\ — n\. 

(F5) F^{m) = r \{ m G S3 — 713 or m belongs to a connected component 
of 713 — 713 adjacent to a member of m G S3 — 713; and F?,(m) = p 
if m G S3 — 713 or m belongs to a connected component of n\ — n\ 
adjacent to a member of m G S3 — 713 . 

(F6) Fj(m) = s C C 2 where s is an arbitrary 1-dimensional subspace of C 2 
for all m in any connected component of nj U 7T 2 — nj n nj other than 
the ones mentioned in (3),(4),(5). 
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In terms of these data 

chx(J-) = -2A - Aj - A 2 - A 3 , 

ch 2 (JT) = A 2 /2 + (A + £» 2 /2 - £>Ay " E # 4 

i i<j i,j 

Given T as above, we define the rank 1 torsion-free T-equivariant sheaves 
L\, £2, £3, to be the maximal subsheaves of T respectively with 

u = 0, v = A 2 , w = A + A3, generated by p 

u = Ai, v = 0, w = A + A3, generated by q 

u = Ai, v = A2, w = A, generated by r. 



We are only interested in the case where the Hilbert polynomial of T is 
Vim) = m 2 + 3m + b, so we must have 

A a + A 2 + A 3 = -2A, 

and 

& = a 2 -£a,a ; -£#4 

! '<; ! V 

Then one can see that J 7 is (semi)stable if the Hilbert polynomials of C\, C2, £3 
are less than (less than or equal to) P/2 (see HKoo08l Proposition 3.19]). Us- 
ing this set up we have 

Proposition 3.6. Suppose T is a rank 2 semistable T-equivariant sheaf on P 2 
with Hilbert polynomial P(m) = m 2 + 3m + b given by the data above. Then 
the contribution of T to the pair invariants PI„(X;P), denoted by P7„(J r ) is as 
follows: 

(1) If T = Izj © Zz 2 , where Xz 1 ,Xz 2 are the ideal sheaves of the T -invariant 
0-dimensional subschemes Z\, Z 2 C P 2 such that the Hilbert polynomials 
ofXz v Xz z aye equal to P/2 then 

'p(n)(P(n)-2)/8 ifZ x = Z 2 , 
P(n) 2 /4 ifZx £ Z 2 . 



(2) If T is indecomposable and strictly semistable then Pl n [T) = P(n)/2. 

(3) IfT is stable then Pl n {T) = P{n). 



Proof. Given T as in the statement of Proposition 3.6 Pl n {T) is equal to 
the Euler characteristic of the space of T-equivariant sections of Tiyi) satis- 
fying the stability condition for the pairs. This number can be obtained by 
counting the number of the lattice points in the toric description of T given 
as above corresponding to these sections. If m is such a lattice point and 
Fj(m) is 2-dimensional then m corresponds to a P 1 worth of T-equivariant 
sections and hence, m must be counted with multiplicity 2 to account for 
the Euler characteristic of P 1 . If Fj(m) is 1-dimensional then m is counted 
with multiplicity 1. □ 
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Before stating the next theorem we give the following definition: 

Definition 3.7. Let D\, D2 G Z + be positive integers and let n\, tt 2 , . . . , 7T 2 G 
be 2d partitions. We say that the 8-tuple 



{D 1 ,D 2 ,n\,Til,...,7zl) £Z 2 + x 



is P-admissible if 

b = -D1D2 -#7z\-#n 2 #?4, 

b/2 = -#n\ - #tt| - #n\ n n\. 

Define 

C(&) = (3/2) • #{Q G Z+ x <^ 6 | Q is P-admissible} . 







Using Lemma 3.2 and Proposition 3.6 we prove the following result 
evaluating the DT invariants of X corresponding to the rank 2 torsion-free 
sheaves on P 2 : 

Theorem 3.8. Let P(m) = m 2 + 3m + b where b G Z, and to .A/f(P 2 ;P) C 
M. (P 2 ; P) be the open subset of the stable sheaves. 

(1) If bis an odd number then DT(X;P) = DT(X;P) = x(M(P 2 ;P)). 

(2) Ifb is an even number then 

DT(X;P) =^(Hilb^ /2 (P 2 ))/4-x(7W s (P 2 ;P)) - C(b). 

Proof. (1) is already proven (see the discussion at the beginning of this sec- 
tion). We now prove (2). By Lemma [3T2| 

DT(X;P) = ^(Hilb- b/2 (P 2 )) 2 • P(n)/8 - PJ„(X;P)/P(n). 

PJ„(X;P) is the sum of the contributions of the T-equivariant semistable 
sheaves to the Euler characteristics evaluated in Proposition 3.6 The con- 
tribution of the split sheaves is given by: 

^(Hilb- fc/2 (P 2 )) • P{n){P{n) - 2)/8 
+ ^(Hilb- b (P 2 ))(^(Hilb- b/2 (P 2 )) - l)/2 • P(n) 2 /4. 

The contribution of the stable sheaves is equal to #(.M S (P 2 ;P)) • P(n) for 
which a combinatorial formula is given in Remark 3.11 Finally from the 
toric description given above, the contribution of the indecomposable strictly 
semistable sheaves to PI n (X;P) is 

3# {O G Z 2 x ^ 6 | Q is P-admissible} . 

Now the formula in the theorem is obtained by adding all these contribu- 
tions. 

This also finishes the proof of Theorem [2] □ 
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DT(X;P) is in general a rational number in the presence of semistable 
sheaves. Joyce and Song in | |JS11| Section 6.2] define the corresponding BPS 
invariants denoted by DT(X; P) by the following formula: 

DT(X;P) = £ ^DT{X;P/d). 

d>l,d\P(m) 

Joyce and Song conjecture that DT(X;P) is an integer. In the case that 
there are no strictly semistable sheaves with Hilbert polynomial P we have 
DT(X;P) = DT(X;P). 



Corollary 3.9. Using the notation of Theorem 3.8 we assume that b is an even 
number then 

DT(X;P) = -x(M s (F 2 ;P)) 

- (3/2)# {Q E Z 2 x ^ 6 | Q is P-admissible} . 

□ 

In particular to show DT(X; P) G Z one needs to show that 

#{OeZ 2 + x^ 6 |ais P-admissible} G 2Z. 

Example 3.10. In this example we work out the cases b = 0, b = —2, and 
b = —4. In the first two cases we provide more details to make the proof of 
Theorem 13.81 clearer. 

• b = 0. In this case the only semistable sheaf with Hilbert polynomial 
P{m) = m 2 + 3m is isomorphic O © O. Therefore, by Proposition 



3.6 part (1) we have PI n (X;P) = P(n)(P(n) - 2)/8, and hence by 



Lemma 3.2 and noting that DT(X; P/2) = 1 we get 
DT(X;P) = P(n)/8 - (P(n) - 2)/8 = 1/4 

in agreement with the result of Theorem |3.8| We can easily see that 
DT(X; P) = 0. 

& = — 2. Suppose that J 7 is a T-equivariant sheaf with Hilbert poly- 
nomial m 2 + 3m — 2. Then either T = I% 1 © Iz 2 as i 11 Proposition 



3.6 part (1), or J 7 is strictly semistable and indecomposable as in 
Proposition |3 .6 1 part (2). So we have 

P;„(X;P) =3P(n) 2 /4 + 3P(tt)(P(n) - 2)/8 + 6P(n)/2. 

The first term is the sum of the contributions of Iz t © ^z 2 where Z\, 
Z2 are two distinct T-fixed points of P 2 . The second term is the sum 
of the contributions Xz © 1z where Z is a T-fixed point of P 2 , and 
the last term is the sum of the contributions of the T-equivariant 
strictly semistable indecomposable sheaves which are determined 
by the following data: 

Ax = 2,A 2 = A 3 = I, #tt{ = l,#n) = £ (1,1), 
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Ai 


= 2,A 2 


= A 3 


= 1, 


#7T 2 


= h#7t] 


= (i,;)^(2,l), 


A 2 


= 2,A 1 


= A 3 


= 1, 




= h#n] 


= (i,;) ^ (1,2), 


A 2 


= 2,Ai 


= A 3 


= 1, 


#77r| 


= h#n] 


= (/,;) ^ (2,2), 


A 3 


= 2, A, 


= A 2 


= 1, 


#4 


= i,#/r; 


= (i,;) ^ (1,3), 


A 3 


= 2, Ai 


= A 2 


= 1, 


#4 




= (i,;) ^ (2,3), 



Now using the fact that DT(X; P/2) = 3 from ((Mj), by Lemma |32 
we get 

DT(X;P) = 9P(n)/8 - 3P(n)/4 - 3(P(n) - 2)/8 - 3 = -9/4 

in agreement with the result of Theorem |3.8| We can easily see that 
DT(X; P) = -3. 

• b = -4. In this case x{M s (X;P)) = 36 by Remark|34l] x(Hilb 2 (P 2 ) ) = 



9 by ( n4j ). It can be seen that the elements of the set 

{Q G Z 2 x ^ 6 | Q is P-admissible} 
are as follows: 

(1,1,5,0, , , , ), (i,i, B, ,□,,,), (i,i, B, ,,□,,), (i,i, B, ,,,□,), 
(i,i, ,□, , , ,B), (i,i, , ,□, , ,B), (i,i, , , ,., ,B), (i,i, , , , ,n,B), 

(1,1, m,n, , , , ), (1,1, m, ,□,,,), (1,1, m, ,,□,,), (1,1, m, ,,,□,), 
(1,1, ,□, , , ,m), (i,i, , ,□, , ,m), (i,i, , , ,□, ,m), (1,1, , , , ,n,m), 
(1, ,,,□), (1,1, □, ,□, , ,□), (1,1, □, , ,□, ,□), (1,1, □, , ,,□,□), 

(2 1 B W2 1 m ) (2 1 u) (2 1 □□") 

fi 2 B wi 2 m ) n 2 Bwi 2 



(2.1 □ □) (1 2 □ CP 



So by Theorem|3.8|Dr(X;P) =9/4 - 36 - 3/2 • 30 
DT(X; P) 



-81 by Corollary 3.9 



-315/4. Also 





Remark 3.11. In BKoo09i Corollary 4.2] the Euler characteristic of 

M ls (X;P) C M S (X;P), 

the moduli space of ^/-stable rank 2 sheaves, is computed. Let P(m) 
m 2 + 3m + 2 + b be the Hilbert polynomial corresponding to a = then 



£*(.Mf iS (P 2 ;m 2 + 3m + 2 + &))£/ fc = 



00 00 n mn+m+n 



y y 1 

nn> (i-^) 6 titi 1 -? 



m+n 
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In order to find x(M s (P 2 ;P)) appearing in Theorem |3.8| we need to add 
to the formula above the contribution of the stable sheaves which are not 
/^-stable. Here 

M s (P 2 ;m 2 + 3m + 2 + b) C M(F 2 ;m 2 + 3m + 2 + b) 

is the open subset of the stable sheaves (see HKoo09l ). Using the toric 
method above one can see that this contribution is given by 

3# {o = (Di, D 2 , n\,..., n 2 ) G X\ x ^ 6 | Q satisfies |15}} . 

(15) b = -DlD 2 — #7r| -#7T? #7T§, 

b/2 > -#7l{ - #7r| - #7T^ □ 7lf. 

Note that if y is the number of the connected components satisfying (F6) 
in the toric description of a stable toric sheaf T , then T belongs to a T- 
fixed component of the moduli space of stable sheaves isomorphic to (P 1 )K 



The combinatorial expression above, satisfying ( 15 1, accounts for such non- 
isolated components in a subtle way. 

Remark 3.12. It would be very interesting to investigate the possible mod- 
ular properties of the generating series of the invariants 

DT(X; m 2 + 3m + b) or DT(X; m 2 + 3m + b). 
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Appendix A. Relative moduli space of sheaves 
In this appendix we re view part of the construction of Li and Wu in 



2.3 



Let q : W — > A 1 be a good degeneration 



[LW11J needed in Section 
of the projective threefolds, i.e. 

i) W is smooth, 

ii) all the fibers except 7r _1 (0) are smooth projective threefolds, 

iii) 5T _1 (0) = W\ Ud W 2 where W,- is a smooth threefold, D C W; is smooth 
divisor, and 7r _1 (0) is a normal crossing divisor in W. 

Li and Wu in [LWllj construct the Artin stack of expanded degenerations 



2U - 



■> w 



-> A 1 . 



Away from r 1 (0) the family 2U is isomorphic to the original family 

W\7r _1 (0) -> A^O. 

The central fiber n~ l (0) of the original family W — >■ A 1 is replaced in 2TJ by 
the union over all k of the £:-step degenerations 

W[k] = Wi U D P(0 No) U D P(0 © N^) U D ■ ■ ■ U D P(0 A#) U D W 2 

together with the automorphisms C*' c induced from the C*-action along 
the fibers of the standard ruled variety P(C Np). Similarly, for a pair 
of a smooth projective threefold Y and a smooth divisor D C Y, the Artin 
stack of expanded degenerations 



2) 



21 



-> spec C 

is constructed in IILW11| using the fc-step degeneration 

Y[k] = Y U D P(0 © No) U D P(0 © N^) U D • • • U D P(0 © N~l 
together with the automorphisms C*^ as above. 



Definition A.l. f lLWlll Definition 3.1, 3.9, 3.12]) Let J" be a coherent sheaf 
on a C-scheme T of finite type, and suppose that Z C T is a closed sub- 
scheme. J 7 is called normal to F if Tor 1 T (.F, Oz) = 0. A coherent sheaf F 
on W[fc] is called admissible if J 7 is normal to all singular loci D ; - = D of 
W[k]. Similarly, a coherent sheaf T on Y[k] is called relative if J 7 is normal 
to all the singular loci D, = D as well as the divisor at infinity of the last 
copy of P(C©N^). ' 
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Suppose that V is a locally free sheaf on W (respectively on Y), and 0(1) 
be an ample invertible sheaf on W — > A 1 (respectively Y). Li and Wu 
construct the Quot schemes Quot^ e (respectively Quoty' p ) of quotients 
(p : p*V — > T on W[k] (respectively on Y[k]) satisfying 

i) J 7 is admissible (respectively relative), 

ii) <p has finitely many automorphisms covering the automorphisms C* k , 

iii) The Hilbert polynomial of T with respect to p*0(l) is P. 

Moreover, they show that Quot^ c (respectively Quoty'^) is a separated, 
proper over A 1 (respectively separated, proper), Deligne-Mumford stack 
of finite type ( ||LWlll Theorems 4.14 and 4.15]). 

Definition A.2. For any C-point (p : p*V — > T of Quot^ € (respectively 
Quoty'^), we say that J 7 is Gieseker (semi)stable if 

i) the restriction of J 7 to W, (respectively Y) is Gieseker (semi)stable, 

ii) the restriction of J 7 to each of the components P(0 © Np) is Gieseker 
(semi)stable, 

iii) the restriction of T to each of the singular loci D, = D of W[k] is 
Gieseker (semi)stable (see Definition |0Jj. 

Suppose now that the Hilbert polynomial P is such that for any J 7 as above 
the Gieseker semistability implies Gieseker stability. Let R be the open 
substack of Quot^/g. (respectively Quoty'^) consisting of the quotients as 
above such that T is Gieseker stable and the induced map H°(V(N)) — > 
H°(J"(N)) is an isomorphism. We define the moduli stack 9Jt(2B/£;P) 
(respectively 9Jt(Y/ D; P)) by taking the quotient stack of R by the natural 
action of GL(N), where 

V = ©fjf ) 0(-N) 

for some fixed N ^> 0. By our assumption all C-points of 9Jt(2U/ £; P) (re- 
spectively 9JT(Y/ D; P)) have the stabilizer group C*. We define the relative 
moduli space of stable sheaves 

M{W/£;P) (respectively M(Y/D;P)) 

by rigidifying9Jt(2B/e:;P) (respectively Wl{Y/D;P)) IIAGV08L 
M (2H/C; P) (respectively M(Y/D; P)) is then a separated Deligne-Mumford 
stack of finite type. Moreover, assume that the Hilbert polynomial P is such 
that for any C-point (p : p*V — > T of Quot^ c (respectively Quoty'^), the 
purity of J 7 implies that J 7 is Gieseker stable, then Ai(W/€;P) (respec- 
tively M. (Y/ D; P)) is proper over A 1 (respectively proper). 

From now on we assume that A4 = C; P) (respectively M. = 

Ai(Y /D; P)) can be constructed as in Definition |A.2 If for any C-point T of 
M we have Ext 3 (J 7 , F) Q = then, by the result of IIThoOOllHTTOllMPTlOI . 
there is a perfect obstruction theory on A4 relative to € (respectively 21) 
given by 



INVARIANTS OF 2-DIMENSIONAL SHEAVES 



39 



(16) (T^Rn M ^RJ^om(W,F))) [-1], 



where F is the universal sheaf. 
This defines a virtual cycle 

[M(W/C;P)] vir e A n+l (M{W/€}P)) 

(respectively [M(Y/D;P)] vir E A n (M(Y/D;P))), where n is the rank of 
the relative obstruction theory above. Moreover, the restriction of the vir- 
tual cycle [M (2B/<£; P)] vir to a general fiber W t of q : W -> A 1 is the virtual 
cycle 

[M(W t ;P)] vir e A n (M(W t ;P)). 
The degeneration formula of Li-Wu follows from the naturality of the vir- 
tual cycle [.M(2B / '£;P)] mr '. In case n = and assuming properness (see 
Definition [A^, this can be written as 

(17) deg[M(W t ;P)] vir = deg[X(2U+/^;P)] I ' i> / 

where 

2H$ >2U 



eS ><t 

is the stack of node marking objects in 2Uo as defined in ||LWlll Section 2.4]. 

Suppose now that the moduli space of stable sheaves on the smooth sur- 
face D, containing all the restrictions J-"|d (where T is as above) with the in- 
duced Hilbert polynomial, is smooth. Then, the virtual cycle [M. (2Uo/ £j; P)] 1 
on the right hand side of (T7\ can be expressed as a finite weighted sum of 
the products 

[M{W x /V;P\)\ vir x [M(W 2 /D;P 2 )] vir 
over all possible decompositions of the Hilbert polynomial P, provided that 
all such [M{Wj/D;Pi)] vir exist (see IlLWTlllMTrTOl ). 

In case the dimension of the virtual cycle [A4(Y / D; P)] mr is zero (i.e. 
when n = in above discussion), and Ai (Y '/ D; P) is proper, we define the 
relative DT invariant 



DT{Y/D;P) = deg [M(Y / D; P)] vir . 
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Appendix B. 

Counting curves in surfaces in Calabi-Yau 3-folds 
by Amin Gholampour, Artan Sheshmani and Richard Thomas 

Fix a Calabi-Yau 3-fold X with H l (X,O x ) = 0. In this append ix we 
describe a way to count pairs of the form 

(18) ZcH in X, 

where H is a suitably ample hypersurface and Z is a one-dimensional sub- 
scheme of H. While we demand that H be pure of dimension 2 (i.e. a 
divisor in X), the subscheme Z need not be pure, so is in general the union 
of a curve and a 0-dimensional subscheme. 

B.l. Outline. Since H^X, O x ) = 0, all deformations of H are in the same 
linear system 

\H\ :=P(H°(Ox(H))). 
This carries a universal hyperplane 

(19) H C X x \H\ 

\H\. 

The data ( |T8"| are naturally parameterized by the relative Hilbert scheme of 
1-dimensional subschemes of the fibers of the family ( [T9| . That is, if we fix 
j6 £ H2(X, Z) and «6Z, then the moduli space of pairs ( flB"} is the relative 
Hilbert scheme 

(20) Hi% n (ft/|H|) 

of 1-dimensional subschemes Z of the universal hyperplane whose push- 
forward to X has fundamental class j5 = [Z] and holomorphic Euler char- 
acteristic n = x(Oz)- 

We will produce a symmetric perfect obstruction theory on |20| by con- 
sidering it as a moduli space of sheaves of the form 

(21) Uz G Coh(X). 

Here i : H <—> X is the inclusion, and Iz denotes the ideal sheaf of Z con- 
sidered as a subscheme of H. So |2T] | is a torsion sheaf on X with rank 1 
on its 2-dimensional support H. When Z C H is a Cartier divisor, it is the 
pushforward of a line bundle on H. 

There are two obvious problems with forming a moduli space of sheaves 
such as ( |2"I) : 

• They need not be (Gieseker) semistable when H is reducible or nonre- 
duced. 

• Deformations of ( |2T| | need not be sheaves of the same form. They 
could be arbitrary torsion sheaves of the same topological type, like 
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the pushforward z*L of a general line bundle on a hyperplane H 
rather than one which is a subsheaf of Oh- 

We circumvent these by using certain Joyce-Song pairs IJSllj. These 
come with a different notion of stability which gets round the first prob- 
lem. And for H>0, they allow us to see that deformations of pi) are also 
push forwards of ideal sheaves, so that Yti\bp in (H/\H\) is indeed an open 
and closed subscheme of the stack of all coherent sheaves on X. 

Our Joyce-Song pairs are of the form 



(l z , s e H°(Z Z (H))), 



where Z and H are as before, but Iz denotes the ideal sheaf of Z when con- 
sidered as a subscheme of X (rather than H: we always use straight 7s to 
denote ideal sheaves on H and curly Is for ideal sheaves on X). Since the 
torsion-free rank-1 sheaf Iz is automatically Gieseker stable, the only fur- 
ther stability condition we need is that s should be nonzero (see Definition 
|0.1|. Therefore we get an injection 



(22) 0{-H) Iz 

whose cokernel i*Iz is the torsion sheaf pi) . More globally we will show 



(Proposition |B.1| that the observation that the Joyce-Song complexes (j22J) 
are quasi-isomorphic to the torsion sheaves plj ) gives an isomorphism be- 
tween 

• an open and closed subscheme of the moduli space of Joyce-Song 
pairs $22) , and 

• the relative Hilbert scheme p0| parameterizing the sheaves pl"| l. 

Finally, for H 3> 0, Joyce and Song show that the deformation theory of 
the complex p2| in D(X) gives this space a symmetric perfect obstruction 
theory. Since (|22"|| is quasi-isomorphic to i*Iz £ D(X), this shows that the 
deformation theory of the sheaf i*Iz indeed endows Hilb« n (%/|H|) with 
a symmetric perfect obstruction theory. 

B.2. Details. Fix a Calabi-Yau 3-fold X with H l {X, O x ) = and an ample 



class H. A Joyce-Song stable pair [ JS1 1 1 is a pair 



E, s G H°(E(H)) 



such that 



• E is coherent sheaf on X, Gieseker semistable with respect to H, and 

• s is a section which does not factor through any destabilizing sub- 
sheaf of E. 

A family of such pairs over a base scheme B is a sheaf S over X x B, 
flat over B, and a section of £ <g) n\0 (H) such that the restriction of (£, s) 
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to any fiber X x {b} is a stable pair in the above sense. Fixing a Chern 
character c £ H ev (X, Q), there is a projective scheme 

representing the moduli functor which assigns to a scheme B the set of 
isomorphism classes^] of Joyce-Song stable pairs over B. 

Forgetting the section s and passing to the S-equivalence class of the 
sheaf E gives a morphism 

J c (X) — > M C (X) 

to the moduli space of Gieseker semistable sheaves E of Chern character c. 
We take 

c= (1,0, -£,-«). 

Then the Hilbert scheme I n (X,B) of ideal sheaves Xz defines an open and 
closed subscheme^] of M C (X) (see for instance the proof of Theorem 2.7 in 
IIPT09I for a careful proof of this fact). Thus its inverse image 

J n (X,B):=J c (X)x Mc{x) I n (X,B) 

is both open and closed in J C (X). It therefore has the same deformation 
theory as J C (X), so we can use the results of Joyce and Song. 

Proposition B.l. J n (X,B) ^Hilb^ n (H/\H\). 

Proof. Hilb^ /)! ('H/ |H|) carries a pair of universal subschemes 

(23) £cft CXxHi%„(ft/|H|). 

Letting i denote the second inclusion, we get the right hand column and 
central row of the following commutative diagram of short exact sequences. 

(24) 

I I 

— > OxxHiib(-H) > X z > ij z > 



0- 


-> C>XxHilb(— W) ~ 


— > CxxHilb — 


I 


->0 






1 


-UOz 








1 




I 

0. 





Isomorphism is meant in the strict sense: two families of stable pairs (£,-, s, ) are isomorphic 
if there is an isomorphism £\ — > £2 taking sj to S2. Stable pairs have no automorphisms so 
there is no need to tensor by line bundles pulled back from the base (as one does to define 
isomorphism for families of stable sheaves, for example: stable sheaves have Aut = C*). 
9 In fact it is all of M C (X) when H 2 (X,Z) is torsion-free. 
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Filling in the diagram gives the top row, producing the flat family of stable 
pairs 

(25) O ^Z z {U-n x H)®n x O{H) 

over the base Hilb« n (%/|H|). This is classified by a map from the base to 
the moduli space of stable pairs: 

(26) mbfa(H/\H\)-^J n (X,p). 

Similarly, X x J n (X, /3) carries a universal stable pair 

(27) O — >£®tc* x O(H), 

flat over J n (X, ft). Since the restriction of £ to each X-fiber is torsion-free of 
rank 1, its double dual £ vv is locally free by [Kol90, Lemma 6.13]. There- 
fore it defines a map from J„(X,/3) to Pic(X) which takes closed points to 
the trivial line bundle O x . But H l (X,O x ) = so Pic(X) is a union of 
discrete reduced points and the map is constant. Pulling back a Poincare 
line bundle shows that £ vv is the pullback n*jL of some line bundle L on 
J„(X, |S). Therefore £ C £ vv must take the form 

(28) £ ^ 7Z}L®1 Z 

for some subscheme Z C X x } n (X, j6). Since £ is flat over J n {X, j6), so Z 
must be too. 

Composing the section ( |27| | with £ — > £ vv gives a section 

(29) XxUX/P) -^0(H)®L, 
nonzero on each X-fiber by stability. Its zero locus is a divisor 

H e \0(H)mL\, 

giving a classifying map /„(X,/5) — > \H\ such thatthe pullback of Xx \ H \(H) 
is O(H) M L. Since the section (|29) factors through ^ it follows that 

zcn, 

giving us a classifying map from our base J n (X,f5) to Hilb^ / „('H/ |H|). By 
inspection it is the inverse of the map |26|. □ 



Definition B.2. Assume that f> G H2(X;Z) and «6Z are given. We say H 
is sufficiently positive with respect to /3, n if 

(30) H i (X,l z (H))=0 

for i > for any ideal sheaf lz £ In (X, /3). For fixed (/3, n) the ideal sheaves 
2z G 7 n (X, /S) form a bounded family, so pO) is satisfied for all H suffi- 
ciently positive. 
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Corollary B.3. Suppose that H 1 (X, Ox) = and f3, n, H satisfy ( |30| . Then, us- 
ing the same notation as in ( |23] l, the deformation-obstruction theory of the sheaves 
U l z (|2T) given by IIHT101 Section 4.4], 

(31) TWRn m RUom(iJ z JJ z )[2]^^ nabanmi 
defines a symmetric perfect obstruction theory on Hilb^„ (H/ \H\ ). 



In (31 1, tch denotes the projection from X x Hilb^ <n ('H/|H|) to its second 
factor. We recall the definition of the arrow 1HT101 Section 4]. Project the 
Atiyah class 

At(z*Iz) G Ext 1 (UI Z , i*Iz <8>LxxHilb) 

to 

Ext 1 (iJz, iJz ® 7r|f]L Hilb ) 
and consider the resulting element as a map 

(32) TTHLffiib — > RHom(iJ z ,Uz)[l]. 
By adjunction this induces 

(33) L^ ilb Rn m RUom(iJ z ,iJ z )[l}. 

Next we project to the T-° truncation of the right hand side. In turn this 
receives a map from the truncation t^ ,1 \ and in MHT10[ Section 4.4] it is 
shown that the map from L^iib ^ s uniquely to it. Dualizing and using 
Serre duality down the fibers of gives pi) . 
Corollary B.3 completes the proof of Theorem|3] 



Proof of Corollary [BJ] For H 3> 0, Joyce and Song[JSll , Theorem 12.20] give 
a symmetric perfect obstruction theory on / C (X): 

(34) T^Rnj^R-HomCl' ,T)[2] — ► L /e(x) . 

Here n } is the projection X x J C (X) -> J C (X) and I* G D(X x / C (X)) is the 
2-term complex 

Gxxj c (X)(-rtxH) — > & 
defined by the universal stable pair O —¥ £ (g> n x O{H). (From now on we 
suppress the n x .) The arrow in ( |34| is the composition of Serre duality for 
R7Tj t RHom(r, I*) and the Atiyah class of I*, just as in |33). 

We restrict their result to Hilb^ / „('H/|H|) using Proposition B.l Their 



precise H ^> condition is that the sheaves in their stable pairs are ir- 
regular; this becomes the cohomology vanishing condition |30) . The first 
row of the diagram p4| gives the quasi-isomorphism 

r®o{H-u) = {o XxHa ^ n{n/m (-n)^iz} ^ uiz. 

Using the line bundle 0(1) pulled back from the projective space |H|, we 
have the isomorphism 

0(H) = 0(H)MO(l). 
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In particular, I* ~ u lz ("H — H) =ul z {l), so (|34]l can be written 

T^Rn Ht RHom(Uz(l) f Uz(l))[2] — > Lrab M (^/|H|). 

Again the arrow is given by the composition of Serre duality and the Atiyah 
class of z* J.2(l). 



This is almost identical to (3lJ 33j except for the twist by 0(1). Now 

At(iJz(l)) = At(iJ z )+idf, Jjr ®At(0(l)) 

in Ext 1 (t J 2 (l), jj 2 (l) (8) 7T^L Hi ib) = Ext 1 (ij z , Uz ^HUb)- That 
is, tensoring with 0(1) changes the map ( [32] ) by addition of the following 
composition 

^Miib ► 0xxHiib[l] >RUom{iJ z ,iJ z )[l}. 



Therefore ( |33| changes by addition of the composition 

L Hiib At( ° (1)) > R7TH*0xxmb[l] -^-fr Rn Hif Rnom(iJ z ,iJ z )[l\. 

The truncation procedure gives unique lifts to the t! 0,1 ! truncation of the 
central and right hand terms. But our H x (X,O x ) = condition means that 
rl ' 1 ] applied to the central term is zero. □ 



Thus we can define invariants counting the pairs (18 1. Then we will re- 
late them to the MNOP invariants I^ n counting subschemes Z C X. 

Definition B.4. Let X be a Calabi-Yau 3-fold with H 1 (X, O x ) = and sup- 
pose that H is sufficiently positive with respect to /3, n in the sense of Defi- 
nition E21 

The perfect obstruction theories of Corollary B.3 and |ThoOO] respec- 
tively endow Hilb^ / „('H/ |H|) and I„(X,/5) with virtual cycles of dimension 
zero. We define 

Nf n := / 1, lp, n := [ I- 

Since these obstruction theories are symmetric, by HBehQ9H the invariants 
can also be written as weighted Euler characteristics 

(35) Ng, = X (Wbfa(H/\H\) tVim ^ m/m ), 

k,n = X[ I n(X,P),V In ( X ,p)) t 

where Vm denotes the Behrend function [Beh09| of a scheme M. 

By the usual arguments IIHT101 plT] IThoOOl these invariants are un- 
changed under smooth deformation of X. 
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B.3. Relation to MNOP invariants. There is an obvious forgetful map 

(36) mb Pin (H/\H\)—^I n (X t p), (ZCH)4Z. 

The fiber over Z C X is P(H°(X Z (H))). More globally we have the follow- 
ing result. We use the notation 

ZcXx/„(X^)Al B (X,)5) 
for the universal subscheme and the projection to the second factor. 



Lemma B.5. Supposing again that f5, n, H satisfy ( pO} , then the map (|36) is the 
projective bundle 

Hilb« n (^/|H|)^P(7r I# X 2 (H)). 



Proof. The cohomology vanishing condition ( |30| ensures that rti*Xz (H) is 
a vector bundle. Its pullback to V{ni*X z {H)) carries a tautological sub- 
bundle 

0{-l) 7t h Xz(H), 
where again we have suppressed the pullback map. By adjunction we get 

and so a nowhere vanishing section of Xz(H) Kl 0(1) = Xzi^H)- This is a 
family of stable pairs over the base T{ni*X z (H)), classified by a map 

(37) W{n H X z {H)) — ► J»(X,0) S H%,„ (ft / 1 H | ) . 

Conversely X x Yti\bp in (H/ |H|) carries the universal family of stable 
pairs (25 1. Twisting by 0{— 1) (pulled back from \H\) gives 

O(-l) -^Tz(H). 
Pushing down to Httbfi n (%/|H|) gives 

(38) O(-l) 7T H ,X 2 (H). 

Now tth*Xz(H) on Hilb^„("H/|H|) is the pullback of the bundle tz^Xz(H) 



on 7„(X,/3) via the map ( |36| . Therefore the line subbundle (38 1 is classified 
by a map EHIb^(?£/|H|) -> P(7ri*Xz(H)), the inverse of ||37). □ 

Corollary B.6. For H-^X, C?x) = and fi,n,H satisfying f3"0| l, £/ze invariants 
of Definition \B.4\ satisfy 

wfoere c = c(/3, n, H, X) z's £/ze topological number 

c = X (MH)) = J x (^H 3 + H 2 td 2 (X)-H-^j-n. 
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Proof. By Lemma B.5 the map ( |36| ) 

mbp rn (H/\H\) -^I n {X,p) 

is smooth of relative dimension x{%z(H)) — 1. Therefore by [Beh09, Propo- 
sition 1 .5(i)], the Behrend function of Hilbfl n (H / 1 H\ ) is the pullback of that 
of J„(X,/3), multiplied by (-l)x(z(H))-i. since the fibers of the map d36l) 



have Euler characteristic x{%z(H)), the formulas |35| give the result. □ 



Corollary B.6 finishes the proof of TheoremHI 



B.4. S-duality and modularity. Our motivation for defining the invariants 
N¥ n was to try to understand how to define the " supersymmetric BPS in- 
variants associated to D4-D2-D0 systems" studied by string theorists [GSY07, 
IGY071 |OSV01| . Their S-duality conjecture is that the generating series of 
these putative BPS invariants should be modular. 

Most of the examples of D4-D2-D0 systems studied in [GY07, Sections 



2.1-2.6] are of the form (21 1 above, which is what led to our definition. 
However it seems that in general one should count all (semi)stable torsion 
sheaves of the right topological type supported on hyperplanes, not just 
those of the form ( |2~l] (. (Thanks to Tudor Dimofte and Davesh Maulik for 
discussions on this point.) 

Sometimes all of the sheaves ( |2"T| | are stable, for instance when any mem- 
ber of the linear system \H\ is reduced and irreducible. (The hyperplane 
sections of the quintic threefold have this property, by the Lefschetz hy- 
perplane theorem.) In that case one can think of Nj$ n as the contribution 
of the component Hilbfl n (X) of the moduli space of torsion sheaves to the 
physicists' numbers. 

More generally one would expect to be able to relate our invariants N¥ n 
to invariants counting more general torsion sheaves via a sequence of wall 
crossings^ Ideally these would be in the space of Bridgeland stability 
conditions on D(X), starting from a stability condition that approximates 
Joyce-Song stability for the complexes ( |22| , and ending with one approxi- 
mating Gieseker stability for the quasi-isomorphic sheaves pi) . 

In combination with Corollary |B .6 1 this would express the MNOP invari- 
ants Ia n in terms of modular forms. We plan to return to this in future 
work. 
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